2024-05-09T08:06:26.934 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T08:06:26.935 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663 --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/orig.config.yaml 2024-05-09T08:06:26.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:06:27.079 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698663' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKQK93ZoSJVyh0FgN5mh80cfn3C1829OKLrz8jveeWhoHwFoBuMmkojrO3tluiYU3qV1V2/hLmGx4IyZOfRkIjw= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8VnUGvRzZM416o/3dnvLERa3gz/a+bDRu6IOUS8cu5E8ZaXG3dRCyPokHoOOE2Bmm4qmpFG0H0g9rO9YIPxiE= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfBVrF/8dM0BMe7JkZOQh1kk4Q+/KuTZDMPJQLqoN5Z9DahlRBs7HTqdj3Na221FeXAP71d2L9Di32YsMHc2TM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T08:06:27.079 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T08:06:27.080 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T08:06:27.080 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T08:06:27.082 INFO:teuthology.task.internal:Checking packages... 2024-05-09T08:06:27.107 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T08:06:27.107 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T08:06:27.107 INFO:teuthology.packaging:ref: None 2024-05-09T08:06:27.107 INFO:teuthology.packaging:tag: None 2024-05-09T08:06:27.107 INFO:teuthology.packaging:branch: squid 2024-05-09T08:06:27.107 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:06:27.108 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T08:06:27.333 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T08:06:27.334 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T08:06:27.344 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T08:06:27.344 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T08:06:27.358 INFO:teuthology.task.internal:Saving configuration 2024-05-09T08:06:27.379 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T08:06:27.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T08:06:27.409 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:01:05.446360', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKQK93ZoSJVyh0FgN5mh80cfn3C1829OKLrz8jveeWhoHwFoBuMmkojrO3tluiYU3qV1V2/hLmGx4IyZOfRkIjw='} 2024-05-09T08:06:27.430 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:01:05.445389', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8VnUGvRzZM416o/3dnvLERa3gz/a+bDRu6IOUS8cu5E8ZaXG3dRCyPokHoOOE2Bmm4qmpFG0H0g9rO9YIPxiE='} 2024-05-09T08:06:27.453 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:01:05.444045', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfBVrF/8dM0BMe7JkZOQh1kk4Q+/KuTZDMPJQLqoN5Z9DahlRBs7HTqdj3Na221FeXAP71d2L9Di32YsMHc2TM='} 2024-05-09T08:06:27.453 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T08:06:27.462 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T08:06:27.462 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T08:06:27.462 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T08:06:27.463 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T08:06:27.568 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa6f68cd160>, signals=[15]) 2024-05-09T08:06:27.568 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T08:06:27.579 INFO:teuthology.task.internal:Opening connections... 2024-05-09T08:06:27.580 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T08:06:27.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:06:27.662 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T08:06:27.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:06:27.738 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T08:06:27.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:06:27.816 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T08:06:27.823 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-05-09T08:06:27.841 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-05-09T08:06:27.841 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-05-09T08:06:27.898 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:06:27.900 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-05-09T08:06:27.923 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-09T08:06:27.940 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-09T08:06:27.941 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:06:27.998 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:06:27.999 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:06:27.999 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:06:27.999 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-09T08:06:28.023 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-05-09T08:06:28.040 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-05-09T08:06:28.040 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-05-09T08:06:28.098 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-05-09T08:06:28.098 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-05-09T08:06:28.098 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:06:28.099 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:06:28.100 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-05-09T08:06:28.122 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T08:06:28.135 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T08:06:28.159 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T08:06:28.159 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:06:28.162 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:06:28.164 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:06:28.180 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T08:06:28.191 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T08:06:28.192 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:06:28.219 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:06:28.222 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:06:28.330 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T08:06:28.398 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T08:06:28.417 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T08:06:28.417 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-09T08:06:28.417 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-09T08:06:28.417 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:06:28.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:06:28.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-09T08:06:28.667 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T08:06:28.667 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-09T08:06:28.667 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-05-09T08:06:28.686 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-437.el9.x86_64 2024-05-09T08:06:28.686 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T08:06:28.686 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-09T08:06:28.686 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2024-05-09T08:06:30.792 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-09T08:06:38.287 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 19 MB 00:06 2024-05-09T08:06:42.283 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-09T08:06:43.818 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 16 kB 00:00 2024-05-09T08:06:44.994 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-09T08:06:50.135 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-09T08:06:52.255 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-05-09T08:06:52.256 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-05-09T08:06:52.256 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-05-09T08:06:52.353 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T08:06:52.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T08:06:52.504 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-09T08:06:52.505 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-09T08:06:52.507 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2024-05-09T08:07:05.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09 08:07:05 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-09T08:07:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2024-05-09 08:07:05-- 2024-05-09T08:07:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 13s 2024-05-09T08:07:05.051 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 341M in 12s (28.6 MB/s) 2024-05-09T08:07:05.054 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T08:07:05.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-09T08:07:07.133 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2024-05-09T08:07:07.364 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2024-05-09T08:07:07.476 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2024-05-09T08:07:15.474 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-09T08:07:54.543 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-09T08:07:54.544 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-09T08:07:54.628 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.628 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.628 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.628 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-09T08:07:54.629 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-09T08:07:54.630 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-09T08:07:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-09T08:07:54.632 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-09T08:07:54.633 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-09T08:07:54.634 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-09T08:07:54.635 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-09T08:07:54.636 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-09T08:07:54.637 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-09T08:07:54.638 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-09T08:07:54.639 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-09T08:07:54.640 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-09T08:07:54.641 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-09T08:07:54.642 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-09T08:07:54.643 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-09T08:07:54.644 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-09T08:07:54.645 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-09T08:07:54.646 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-09T08:07:54.647 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-09T08:07:54.648 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-09T08:07:54.649 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-09T08:07:54.650 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-09T08:07:54.651 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-09T08:07:54.652 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-09T08:07:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-09T08:07:54.654 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-09T08:07:54.655 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-09T08:07:54.656 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-09T08:07:54.657 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-09T08:07:54.658 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-09T08:07:54.659 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-09T08:07:54.660 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-09T08:07:54.661 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-09T08:07:54.662 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-09T08:07:54.663 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-09T08:07:54.664 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-09T08:07:54.665 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-09T08:07:54.666 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-09T08:07:54.667 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-09T08:07:54.668 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-09T08:07:54.669 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-09T08:07:54.670 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-09T08:07:54.671 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-09T08:07:54.672 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-09T08:07:54.673 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-09T08:07:54.674 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-09T08:07:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-09T08:07:54.676 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-09T08:07:54.677 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-09T08:07:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-09T08:07:54.679 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-09T08:07:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-09T08:07:54.681 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-09T08:07:54.682 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-09T08:07:54.683 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-09T08:07:54.684 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-09T08:07:54.685 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-09T08:07:54.686 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-09T08:07:54.687 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-09T08:07:54.690 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-09T08:07:54.690 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-09T08:07:54.690 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-09T08:07:54.691 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-09T08:07:54.692 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-09T08:07:54.693 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-09T08:07:54.694 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-09T08:07:54.695 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-09T08:07:54.696 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-09T08:07:54.697 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-09T08:07:54.698 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-09T08:07:54.699 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-09T08:07:54.700 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-09T08:07:54.701 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-09T08:07:54.702 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-09T08:07:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-09T08:07:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-09T08:07:54.705 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-09T08:07:54.706 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-09T08:07:54.707 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-09T08:07:54.708 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-09T08:07:54.709 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-09T08:07:54.710 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-09T08:07:54.711 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-09T08:07:54.713 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-09T08:07:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-09T08:07:54.716 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-09T08:07:54.719 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-09T08:07:54.720 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-09T08:07:54.721 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-09T08:07:54.722 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-09T08:07:54.723 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-09T08:07:54.724 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-09T08:07:54.725 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-09T08:07:54.726 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-09T08:07:54.727 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-09T08:07:54.728 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-09T08:07:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-09T08:07:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-09T08:07:54.731 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-09T08:07:54.732 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-09T08:07:54.733 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-09T08:07:54.734 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-09T08:07:54.735 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-09T08:07:54.736 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-09T08:07:54.737 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-09T08:07:54.738 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-09T08:07:54.739 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-09T08:07:54.740 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-09T08:07:54.741 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-09T08:07:54.742 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-09T08:07:54.743 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-09T08:07:54.744 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-09T08:07:54.745 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-09T08:07:54.746 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-09T08:07:54.747 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-09T08:07:54.748 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-09T08:07:54.749 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-09T08:07:54.750 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-09T08:07:54.751 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-09T08:07:54.752 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-09T08:07:54.753 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-09T08:07:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-09T08:07:54.755 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-09T08:07:54.756 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-09T08:07:54.757 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-09T08:07:54.758 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-09T08:07:54.759 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-09T08:07:54.760 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-09T08:07:54.761 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-09T08:07:54.762 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-09T08:07:54.763 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-09T08:07:54.764 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-09T08:07:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T08:07:54.770 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.770 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-09T08:07:54.794 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2024-05-09T08:07:54.794 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout:for file in vmlinuz System.map config; do 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: fi 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-05-09T08:07:54.795 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.796 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-05-09T08:07:54.796 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-05-09T08:07:54.868 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 68.el9 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8264254 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T08:07:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:07:54.870 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T08:07:54.871 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T08:07:54.872 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:54.872 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T08:07:55.425 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T08:07:56.912 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-05-09T08:07:56.914 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-05-09T08:07:56.958 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.kBa84YC0zv 2024-05-09T08:07:56.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kBa84YC0zv 2024-05-09T08:07:57.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.kBa84YC0zv 2024-05-09T08:07:57.188 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.kBa84YC0zv is 6KB 2024-05-09T08:07:57.218 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.kBa84YC0zv 2024-05-09T08:07:57.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T08:07:57.302 INFO:teuthology.orchestra.run.smithi133.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T08:07:57.303 INFO:teuthology.orchestra.run.smithi133.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T08:07:57.303 INFO:teuthology.orchestra.run.smithi133.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T08:07:57.303 INFO:teuthology.orchestra.run.smithi133.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-09T08:07:57.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-09T08:07:57.476 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-05-09T08:08:27.479 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T08:08:27.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:08:27.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:08:45.858 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-05-09T08:08:54.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:08:54.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:08:57.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-05-09T08:09:09.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:09:09.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:09:12.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-05-09T08:09:27.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:09:27.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:09:31.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-05-09T08:09:49.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:09:49.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:09:49.396 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-05-09T08:09:49.930 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:09:49.930 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-09T08:09:49.930 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-05-09T08:09:49.968 INFO:teuthology.orchestra.run.smithi133.stdout:6.9.0-rc3-g806ec36b2112 2024-05-09T08:09:49.968 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-09T08:09:49.968 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T08:09:49.968 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T08:09:50.970 DEBUG:teuthology.parallel:result is None 2024-05-09T08:09:50.970 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T08:09:50.978 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T08:09:50.978 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:09:50.985 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:09:50.987 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:09:51.025 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T08:09:51.032 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T08:09:51.130 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T08:09:51.130 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:09:51.137 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:09:51.139 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:09:51.228 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T08:09:51.313 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T08:09:51.313 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:09:51.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:51.332 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:09:51.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:51.348 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:09:51.353 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:09:51.374 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:09:51.408 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.425 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.430 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.441 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.487 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.531 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:09:51.535 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T08:09:51.542 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T08:09:51.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:09:51.547 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:09:51.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:09:51.614 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T08:09:51.624 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T08:09:51.625 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:09:51.629 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:09:51.631 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:09:51.663 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:51.771 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:51.863 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:51.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:52.019 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:09:52.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:09:52.091 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:52.134 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:52.188 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:52.272 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:52.316 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:09:52.316 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:09:52.383 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:52.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:09:52.492 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:52.537 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:09:52.579 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:09:52.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:09:52.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-05-09T08:09:52.650 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-09T08:09:52.652 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-05-09T08:09:52.692 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:09:52.724 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:09:52.750 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:09:53.203 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T08:09:53.212 INFO:teuthology.task.internal:Starting timer... 2024-05-09T08:09:53.213 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T08:09:53.223 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T08:09:53.232 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T08:09:53.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-05-09T08:09:53.363 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-05-09T08:09:53.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-09T08:09:53.432 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-09T08:09:53.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-05-09T08:09:53.507 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-05-09T08:09:53.508 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T08:09:53.509 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-05-09T08:09:53.564 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-05-09T08:09:53.564 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-09T08:09:53.591 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-09T08:09:53.591 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-05-09T08:09:53.621 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-05-09T08:09:53.621 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-05-09T08:09:53.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:09:53.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:53.696 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:09:53.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:53.725 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:09:53.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:53.751 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T08:09:53.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:09:53.815 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:09:53.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:09:53.876 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T08:09:53.887 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T08:09:53.902 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T08:09:53.947 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi160.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T08:15:26.110 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-05-09T08:15:26.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:15:26.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:15:26.228 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-05-09T08:15:26.390 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-05-09T08:15:26.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T08:15:26.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:15:26.467 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-09T08:15:26.550 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T08:15:26.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T08:15:26.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:15:26.626 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-09T08:15:26.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T08:15:26.707 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T08:15:26.721 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T08:15:26.721 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:15:26.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:15:26.729 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:15:26.729 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:15:26.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:15:26.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:15:26.759 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:15:26.773 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:15:26.776 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:15:26.790 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:15:26.797 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-09T08:15:26.808 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-09T08:15:26.814 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-05-09T08:15:26.817 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:15:26.821 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:15:26.825 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-05-09T08:15:26.834 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:15:26.838 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:15:26.851 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:15:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:15:26.886 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.891 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.905 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.908 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-05-09T08:15:26.951 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-05-09T08:15:26.988 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:15:27.025 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:15:27.172 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:27.181 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:15:27.183 INFO:teuthology.run_tasks:Running task install... 2024-05-09T08:15:27.191 DEBUG:teuthology.task.install:project ceph 2024-05-09T08:15:27.191 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T08:15:27.191 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T08:15:27.192 INFO:teuthology.task.install:Using flavor: default 2024-05-09T08:15:27.197 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T08:15:27.197 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T08:15:27.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:15:27.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:15:27.200 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:15:27.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:15:27.201 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:15:27.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:15:27.351 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:15:27.352 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:15:27.357 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:15:27.358 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:15:27.426 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:15:27.426 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:15:27.461 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:15:27.461 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:15:27.462 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:15:27.486 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:15:27.487 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:15:27.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:15:27.525 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:15:27.525 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:15:27.550 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:15:27.551 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:15:27.552 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:15:27.603 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:15:27.603 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:15:27.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:15:27.615 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:15:27.625 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:15:27.688 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-09T08:15:27.689 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:15:27.691 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-09T08:15:27.693 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:15:27.693 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:15:27.752 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-05-09T08:15:27.754 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-05-09T08:15:27.821 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:15:27.918 INFO:teuthology.orchestra.run.smithi160.stdout:54 files removed 2024-05-09T08:15:27.930 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:15:27.946 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:15:27.980 INFO:teuthology.orchestra.run.smithi188.stdout:54 files removed 2024-05-09T08:15:28.004 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-05-09T08:15:28.007 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:15:28.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-05-09T08:15:28.332 INFO:teuthology.orchestra.run.smithi133.stdout:54 files removed 2024-05-09T08:15:28.371 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:15:28.642 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 192 kB/s | 83 kB 00:00 2024-05-09T08:15:28.685 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-05-09T08:15:29.068 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-09T08:15:29.152 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-05-09T08:15:29.342 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 156 kB/s | 83 kB 00:00 2024-05-09T08:15:29.402 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-05-09T08:15:29.561 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-09T08:15:29.842 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-09T08:15:29.960 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-09T08:15:30.361 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-09T08:15:30.409 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-09T08:15:31.653 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-09T08:15:32.262 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-09T08:15:32.623 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.1 MB 00:02 2024-05-09T08:15:36.321 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-09T08:15:36.374 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-05-09T08:15:37.487 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-05-09T08:15:37.838 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-05-09T08:15:39.062 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-09T08:15:40.200 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-09T08:15:42.158 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-09T08:15:43.776 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-05-09T08:15:45.236 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-09T08:15:45.331 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-09T08:15:47.641 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:15:47.642 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:15:47.653 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:15:47.654 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:15:47.655 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:15:47.655 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:15:47.657 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:15:47.659 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:15:47.660 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:15:47.721 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-09T08:15:47.741 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:15:47.742 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:15:47.743 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:15:47.744 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:15:47.745 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:15:47.746 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:15:47.747 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:15:47.748 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:15:47.749 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:15:47.750 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:15:47.751 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:15:47.752 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout:Install 148 Packages 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 8 Packages 2024-05-09T08:15:47.753 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:15:47.754 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 225 M 2024-05-09T08:15:47.754 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-09T08:15:48.616 INFO:teuthology.orchestra.run.smithi160.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-09T08:15:48.799 INFO:teuthology.orchestra.run.smithi160.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 4.7 MB/s | 889 kB 00:00 2024-05-09T08:15:48.850 INFO:teuthology.orchestra.run.smithi160.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.8 MB/s | 145 kB 00:00 2024-05-09T08:15:49.175 INFO:teuthology.orchestra.run.smithi160.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 7.3 MB/s | 2.4 MB 00:00 2024-05-09T08:15:49.400 INFO:teuthology.orchestra.run.smithi160.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-05-09T08:15:49.567 INFO:teuthology.orchestra.run.smithi160.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-05-09T08:15:50.001 INFO:teuthology.orchestra.run.smithi160.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 7.8 MB/s | 4.7 MB 00:00 2024-05-09T08:15:50.359 INFO:teuthology.orchestra.run.smithi160.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 11 MB/s | 21 MB 00:01 2024-05-09T08:15:50.418 INFO:teuthology.orchestra.run.smithi160.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 433 kB/s | 25 kB 00:00 2024-05-09T08:15:51.444 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-09T08:15:51.510 INFO:teuthology.orchestra.run.smithi160.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 6.8 MB/s | 10 MB 00:01 2024-05-09T08:15:51.602 INFO:teuthology.orchestra.run.smithi160.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.4 MB/s | 222 kB 00:00 2024-05-09T08:15:51.669 INFO:teuthology.orchestra.run.smithi160.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 406 kB/s | 27 kB 00:00 2024-05-09T08:15:51.744 INFO:teuthology.orchestra.run.smithi160.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 424 kB/s | 32 kB 00:00 2024-05-09T08:15:51.927 INFO:teuthology.orchestra.run.smithi160.stdout:(14/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 7.1 MB/s | 17 MB 00:02 2024-05-09T08:15:51.961 INFO:teuthology.orchestra.run.smithi160.stdout:(15/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 3.4 MB/s | 746 kB 00:00 2024-05-09T08:15:51.995 INFO:teuthology.orchestra.run.smithi160.stdout:(16/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.4 MB/s | 163 kB 00:00 2024-05-09T08:15:52.020 INFO:teuthology.orchestra.run.smithi160.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 2.1 MB/s | 127 kB 00:00 2024-05-09T08:15:52.104 INFO:teuthology.orchestra.run.smithi160.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 4.5 MB/s | 500 kB 00:00 2024-05-09T08:15:52.195 INFO:teuthology.orchestra.run.smithi160.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 493 kB/s | 45 kB 00:00 2024-05-09T08:15:52.288 INFO:teuthology.orchestra.run.smithi160.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.4 MB/s | 135 kB 00:00 2024-05-09T08:15:52.479 INFO:teuthology.orchestra.run.smithi160.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 845 kB/s | 162 kB 00:00 2024-05-09T08:15:52.605 INFO:teuthology.orchestra.run.smithi160.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 2.5 MB/s | 321 kB 00:00 2024-05-09T08:15:52.722 INFO:teuthology.orchestra.run.smithi160.stdout:(23/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 7.6 MB/s | 5.3 MB 00:00 2024-05-09T08:15:52.747 INFO:teuthology.orchestra.run.smithi160.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.1 MB/s | 298 kB 00:00 2024-05-09T08:15:52.781 INFO:teuthology.orchestra.run.smithi160.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T08:15:52.823 INFO:teuthology.orchestra.run.smithi160.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-09T08:15:52.923 INFO:teuthology.orchestra.run.smithi160.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.7 MB/s | 171 kB 00:00 2024-05-09T08:15:53.015 INFO:teuthology.orchestra.run.smithi160.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 346 kB/s | 32 kB 00:00 2024-05-09T08:15:53.115 INFO:teuthology.orchestra.run.smithi160.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.4 MB/s | 142 kB 00:00 2024-05-09T08:15:53.182 INFO:teuthology.orchestra.run.smithi160.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 7.7 MB/s | 3.1 MB 00:00 2024-05-09T08:15:53.591 INFO:teuthology.orchestra.run.smithi160.stdout:(31/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 15 MB/s | 48 MB 00:03 2024-05-09T08:15:53.666 INFO:teuthology.orchestra.run.smithi160.stdout:(32/156): ceph-mgr-dashboard-19.0.0-2515.g96805 6.4 MB/s | 3.5 MB 00:00 2024-05-09T08:15:53.691 INFO:teuthology.orchestra.run.smithi160.stdout:(33/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T08:15:53.717 INFO:teuthology.orchestra.run.smithi160.stdout:(34/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 991 kB/s | 49 kB 00:00 2024-05-09T08:15:53.734 INFO:teuthology.orchestra.run.smithi160.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 392 kB/s | 16 kB 00:00 2024-05-09T08:15:53.770 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:15 2024-05-09T08:15:53.792 INFO:teuthology.orchestra.run.smithi160.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.4 MB/s | 264 kB 00:00 2024-05-09T08:15:53.834 INFO:teuthology.orchestra.run.smithi160.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 7.4 MB/s | 763 kB 00:00 2024-05-09T08:15:53.859 INFO:teuthology.orchestra.run.smithi160.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 273 kB/s | 18 kB 00:00 2024-05-09T08:15:53.951 INFO:teuthology.orchestra.run.smithi160.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-05-09T08:15:53.977 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:15:53.979 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:15:53.999 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:15:54.000 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:15:54.001 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:15:54.003 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:15:54.005 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:15:54.008 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:15:54.009 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:15:54.079 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:15:54.100 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:15:54.101 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:15:54.102 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:15:54.103 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:15:54.104 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:15:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:15:54.106 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:15:54.107 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:15:54.108 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:15:54.109 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:15:54.110 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:15:54.110 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:15:54.110 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:15:54.110 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:15:54.110 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:15:54.113 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:15:54.114 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 8 Packages 2024-05-09T08:15:54.115 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:15:54.116 INFO:teuthology.orchestra.run.smithi160.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 291 kB/s | 80 kB 00:00 2024-05-09T08:15:54.116 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 225 M 2024-05-09T08:15:54.116 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-05-09T08:15:54.169 INFO:teuthology.orchestra.run.smithi160.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 901 kB/s | 195 kB 00:00 2024-05-09T08:15:54.227 INFO:teuthology.orchestra.run.smithi160.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-05-09T08:15:54.277 INFO:teuthology.orchestra.run.smithi160.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-05-09T08:15:54.303 INFO:teuthology.orchestra.run.smithi160.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-09T08:15:54.336 INFO:teuthology.orchestra.run.smithi160.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-09T08:15:54.361 INFO:teuthology.orchestra.run.smithi160.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-09T08:15:54.395 INFO:teuthology.orchestra.run.smithi160.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-09T08:15:54.479 INFO:teuthology.orchestra.run.smithi160.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-05-09T08:15:54.687 INFO:teuthology.orchestra.run.smithi160.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-05-09T08:15:54.713 INFO:teuthology.orchestra.run.smithi160.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 458 kB/s | 107 kB 00:00 2024-05-09T08:15:54.746 INFO:teuthology.orchestra.run.smithi160.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-05-09T08:15:54.771 INFO:teuthology.orchestra.run.smithi160.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-05-09T08:15:54.805 INFO:teuthology.orchestra.run.smithi160.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-05-09T08:15:54.856 INFO:teuthology.orchestra.run.smithi160.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-05-09T08:15:54.881 INFO:teuthology.orchestra.run.smithi160.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 241 kB/s | 18 kB 00:00 2024-05-09T08:15:54.915 INFO:teuthology.orchestra.run.smithi160.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-05-09T08:15:54.965 INFO:teuthology.orchestra.run.smithi160.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-05-09T08:15:54.990 INFO:teuthology.orchestra.run.smithi160.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-05-09T08:15:55.015 INFO:teuthology.orchestra.run.smithi160.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T08:15:55.049 INFO:teuthology.orchestra.run.smithi160.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-05-09T08:15:55.074 INFO:teuthology.orchestra.run.smithi160.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-09T08:15:55.104 INFO:teuthology.orchestra.run.smithi133.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-09T08:15:55.108 INFO:teuthology.orchestra.run.smithi160.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.8 MB/s | 582 kB 00:00 2024-05-09T08:15:55.125 INFO:teuthology.orchestra.run.smithi160.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-09T08:15:55.150 INFO:teuthology.orchestra.run.smithi160.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-09T08:15:55.176 INFO:teuthology.orchestra.run.smithi160.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-05-09T08:15:55.192 INFO:teuthology.orchestra.run.smithi160.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-09T08:15:55.309 INFO:teuthology.orchestra.run.smithi160.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-09T08:15:55.346 INFO:teuthology.orchestra.run.smithi133.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 3.6 MB/s | 889 kB 00:00 2024-05-09T08:15:55.384 INFO:teuthology.orchestra.run.smithi160.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-05-09T08:15:55.412 INFO:teuthology.orchestra.run.smithi133.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.2 MB/s | 145 kB 00:00 2024-05-09T08:15:55.418 INFO:teuthology.orchestra.run.smithi160.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-05-09T08:15:55.477 INFO:teuthology.orchestra.run.smithi160.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-09T08:15:55.502 INFO:teuthology.orchestra.run.smithi160.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-09T08:15:55.569 INFO:teuthology.orchestra.run.smithi160.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-05-09T08:15:55.594 INFO:teuthology.orchestra.run.smithi160.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-05-09T08:15:55.669 INFO:teuthology.orchestra.run.smithi160.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-09T08:15:55.695 INFO:teuthology.orchestra.run.smithi160.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-09T08:15:55.737 INFO:teuthology.orchestra.run.smithi160.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-05-09T08:15:55.762 INFO:teuthology.orchestra.run.smithi160.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-05-09T08:15:55.763 INFO:teuthology.orchestra.run.smithi133.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-05-09T08:15:55.838 INFO:teuthology.orchestra.run.smithi133.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 5.6 MB/s | 2.4 MB 00:00 2024-05-09T08:15:55.846 INFO:teuthology.orchestra.run.smithi160.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-05-09T08:15:55.871 INFO:teuthology.orchestra.run.smithi160.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-09T08:15:55.896 INFO:teuthology.orchestra.run.smithi160.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-09T08:15:55.921 INFO:teuthology.orchestra.run.smithi160.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-05-09T08:15:55.972 INFO:teuthology.orchestra.run.smithi133.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-05-09T08:15:55.988 INFO:teuthology.orchestra.run.smithi160.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-09T08:15:56.022 INFO:teuthology.orchestra.run.smithi160.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T08:15:56.089 INFO:teuthology.orchestra.run.smithi160.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-05-09T08:15:56.114 INFO:teuthology.orchestra.run.smithi160.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 458 kB/s | 42 kB 00:00 2024-05-09T08:15:56.139 INFO:teuthology.orchestra.run.smithi160.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 511 kB/s | 77 kB 00:00 2024-05-09T08:15:56.165 INFO:teuthology.orchestra.run.smithi160.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-09T08:15:56.190 INFO:teuthology.orchestra.run.smithi160.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-05-09T08:15:56.215 INFO:teuthology.orchestra.run.smithi160.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-09T08:15:56.241 INFO:teuthology.orchestra.run.smithi160.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 673 kB/s | 51 kB 00:00 2024-05-09T08:15:56.266 INFO:teuthology.orchestra.run.smithi160.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-05-09T08:15:56.316 INFO:teuthology.orchestra.run.smithi160.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-05-09T08:15:56.341 INFO:teuthology.orchestra.run.smithi160.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-05-09T08:15:56.400 INFO:teuthology.orchestra.run.smithi160.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T08:15:56.425 INFO:teuthology.orchestra.run.smithi160.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 428 kB/s | 36 kB 00:00 2024-05-09T08:15:56.476 INFO:teuthology.orchestra.run.smithi160.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-09T08:15:56.501 INFO:teuthology.orchestra.run.smithi160.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-05-09T08:15:56.543 INFO:teuthology.orchestra.run.smithi160.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-05-09T08:15:56.568 INFO:teuthology.orchestra.run.smithi160.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T08:15:56.597 INFO:teuthology.orchestra.run.smithi133.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 6.2 MB/s | 4.7 MB 00:00 2024-05-09T08:15:56.727 INFO:teuthology.orchestra.run.smithi160.stdout:(100/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-05-09T08:15:56.744 INFO:teuthology.orchestra.run.smithi160.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 255 kB/s | 45 kB 00:00 2024-05-09T08:15:56.769 INFO:teuthology.orchestra.run.smithi160.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-05-09T08:15:56.819 INFO:teuthology.orchestra.run.smithi160.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-09T08:15:56.944 INFO:teuthology.orchestra.run.smithi160.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-05-09T08:15:56.978 INFO:teuthology.orchestra.run.smithi160.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 583 kB/s | 19 kB 00:00 2024-05-09T08:15:57.003 INFO:teuthology.orchestra.run.smithi160.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-05-09T08:15:57.037 INFO:teuthology.orchestra.run.smithi160.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-05-09T08:15:57.054 INFO:teuthology.orchestra.run.smithi160.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-05-09T08:15:57.055 INFO:teuthology.orchestra.run.smithi133.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 9.5 MB/s | 21 MB 00:02 2024-05-09T08:15:57.114 INFO:teuthology.orchestra.run.smithi133.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 441 kB/s | 25 kB 00:00 2024-05-09T08:15:57.121 INFO:teuthology.orchestra.run.smithi160.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-09T08:15:57.138 INFO:teuthology.orchestra.run.smithi160.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 483 kB/s | 49 kB 00:00 2024-05-09T08:15:57.196 INFO:teuthology.orchestra.run.smithi160.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-09T08:15:57.322 INFO:teuthology.orchestra.run.smithi160.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-09T08:15:57.372 INFO:teuthology.orchestra.run.smithi160.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 872 kB/s | 43 kB 00:00 2024-05-09T08:15:57.397 INFO:teuthology.orchestra.run.smithi160.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-05-09T08:15:57.423 INFO:teuthology.orchestra.run.smithi160.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 638 kB/s | 32 kB 00:00 2024-05-09T08:15:57.448 INFO:teuthology.orchestra.run.smithi160.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-09T08:15:57.490 INFO:teuthology.orchestra.run.smithi160.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-09T08:15:57.524 INFO:teuthology.orchestra.run.smithi160.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-09T08:15:57.566 INFO:teuthology.orchestra.run.smithi160.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 848 kB/s | 35 kB 00:00 2024-05-09T08:15:57.591 INFO:teuthology.orchestra.run.smithi160.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-09T08:15:57.616 INFO:teuthology.orchestra.run.smithi160.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-09T08:15:57.683 INFO:teuthology.orchestra.run.smithi160.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-05-09T08:15:57.708 INFO:teuthology.orchestra.run.smithi160.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-09T08:15:57.714 INFO:teuthology.orchestra.run.smithi133.stdout:(10/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 9.6 MB/s | 17 MB 00:01 2024-05-09T08:15:57.734 INFO:teuthology.orchestra.run.smithi160.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-05-09T08:15:57.759 INFO:teuthology.orchestra.run.smithi160.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-05-09T08:15:57.798 INFO:teuthology.orchestra.run.smithi133.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.6 MB/s | 222 kB 00:00 2024-05-09T08:15:57.801 INFO:teuthology.orchestra.run.smithi160.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-05-09T08:15:57.826 INFO:teuthology.orchestra.run.smithi160.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-09T08:15:57.852 INFO:teuthology.orchestra.run.smithi160.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-09T08:15:57.873 INFO:teuthology.orchestra.run.smithi133.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 364 kB/s | 27 kB 00:00 2024-05-09T08:15:57.902 INFO:teuthology.orchestra.run.smithi160.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-05-09T08:15:57.940 INFO:teuthology.orchestra.run.smithi133.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 480 kB/s | 32 kB 00:00 2024-05-09T08:15:57.944 INFO:teuthology.orchestra.run.smithi160.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-09T08:15:57.986 INFO:teuthology.orchestra.run.smithi160.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-09T08:15:58.019 INFO:teuthology.orchestra.run.smithi160.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-05-09T08:15:58.061 INFO:teuthology.orchestra.run.smithi160.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T08:15:58.082 INFO:teuthology.orchestra.run.smithi133.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 5.2 MB/s | 746 kB 00:00 2024-05-09T08:15:58.095 INFO:teuthology.orchestra.run.smithi160.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-09T08:15:58.120 INFO:teuthology.orchestra.run.smithi160.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T08:15:58.154 INFO:teuthology.orchestra.run.smithi160.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 927 kB/s | 31 kB 00:00 2024-05-09T08:15:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.9 MB/s | 163 kB 00:00 2024-05-09T08:15:58.170 INFO:teuthology.orchestra.run.smithi160.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-09T08:15:58.196 INFO:teuthology.orchestra.run.smithi160.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-05-09T08:15:58.229 INFO:teuthology.orchestra.run.smithi160.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-09T08:15:58.240 INFO:teuthology.orchestra.run.smithi133.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.7 MB/s | 127 kB 00:00 2024-05-09T08:15:58.246 INFO:teuthology.orchestra.run.smithi160.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-09T08:15:58.280 INFO:teuthology.orchestra.run.smithi160.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-09T08:15:58.305 INFO:teuthology.orchestra.run.smithi160.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-09T08:15:58.331 INFO:teuthology.orchestra.run.smithi160.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-09T08:15:58.332 INFO:teuthology.orchestra.run.smithi133.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 5.4 MB/s | 500 kB 00:00 2024-05-09T08:15:58.398 INFO:teuthology.orchestra.run.smithi160.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-09T08:15:58.423 INFO:teuthology.orchestra.run.smithi160.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-09T08:15:58.546 INFO:teuthology.orchestra.run.smithi160.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-09T08:15:58.590 INFO:teuthology.orchestra.run.smithi160.stdout:(147/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:00 2024-05-09T08:15:58.624 INFO:teuthology.orchestra.run.smithi160.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-09T08:15:58.667 INFO:teuthology.orchestra.run.smithi133.stdout:(18/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 4.9 MB/s | 10 MB 00:02 2024-05-09T08:15:58.758 INFO:teuthology.orchestra.run.smithi133.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 498 kB/s | 45 kB 00:00 2024-05-09T08:15:58.859 INFO:teuthology.orchestra.run.smithi133.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.3 MB/s | 135 kB 00:00 2024-05-09T08:15:59.000 INFO:teuthology.orchestra.run.smithi133.stdout:(21/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 8.0 MB/s | 5.3 MB 00:00 2024-05-09T08:15:59.021 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T08:15:59.034 INFO:teuthology.orchestra.run.smithi133.stdout:(22/156): python3-cephfs-19.0.0-2515.g96805dcf. 928 kB/s | 162 kB 00:00 2024-05-09T08:15:59.049 INFO:teuthology.orchestra.run.smithi160.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2024-05-09T08:15:59.101 INFO:teuthology.orchestra.run.smithi133.stdout:(23/156): python3-rados-19.0.0-2515.g96805dcf.e 3.2 MB/s | 321 kB 00:00 2024-05-09T08:15:59.135 INFO:teuthology.orchestra.run.smithi133.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 2.9 MB/s | 298 kB 00:00 2024-05-09T08:15:59.141 INFO:teuthology.orchestra.run.smithi160.stdout:(150/156): librados2-19.0.0-2515.g96805dcf.el9. 5.7 MB/s | 3.4 MB 00:00 2024-05-09T08:15:59.185 INFO:teuthology.orchestra.run.smithi133.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T08:15:59.269 INFO:teuthology.orchestra.run.smithi133.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 641 kB/s | 85 kB 00:00 2024-05-09T08:15:59.300 INFO:teuthology.orchestra.run.smithi160.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 7.3 MB/s | 1.8 MB 00:00 2024-05-09T08:15:59.325 INFO:teuthology.orchestra.run.smithi160.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 227 kB/s | 42 kB 00:00 2024-05-09T08:15:59.352 INFO:teuthology.orchestra.run.smithi133.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-09T08:15:59.367 INFO:teuthology.orchestra.run.smithi160.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-05-09T08:15:59.419 INFO:teuthology.orchestra.run.smithi133.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 482 kB/s | 32 kB 00:00 2024-05-09T08:15:59.528 INFO:teuthology.orchestra.run.smithi133.stdout:(29/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 9.0 MB/s | 3.1 MB 00:00 2024-05-09T08:15:59.559 INFO:teuthology.orchestra.run.smithi160.stdout:(154/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-09T08:15:59.561 INFO:teuthology.orchestra.run.smithi133.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.0 MB/s | 142 kB 00:00 2024-05-09T08:15:59.634 INFO:teuthology.orchestra.run.smithi160.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-05-09T08:15:59.667 INFO:teuthology.orchestra.run.smithi160.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-05-09T08:15:59.670 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:15:59.670 INFO:teuthology.orchestra.run.smithi160.stdout:Total 19 MB/s | 225 MB 00:11 2024-05-09T08:15:59.929 INFO:teuthology.orchestra.run.smithi133.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 8.8 MB/s | 3.5 MB 00:00 2024-05-09T08:16:00.012 INFO:teuthology.orchestra.run.smithi133.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 3.0 MB/s | 249 kB 00:00 2024-05-09T08:16:00.087 INFO:teuthology.orchestra.run.smithi133.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 664 kB/s | 49 kB 00:00 2024-05-09T08:16:00.154 INFO:teuthology.orchestra.run.smithi133.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 251 kB/s | 16 kB 00:00 2024-05-09T08:16:00.230 INFO:teuthology.orchestra.run.smithi133.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.5 MB/s | 264 kB 00:00 2024-05-09T08:16:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-09T08:16:00.855 INFO:teuthology.orchestra.run.smithi133.stdout:(37/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 13 MB/s | 48 MB 00:03 2024-05-09T08:16:00.873 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-09T08:16:00.880 INFO:teuthology.orchestra.run.smithi133.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 36 kB/s | 18 kB 00:00 2024-05-09T08:16:01.025 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-09T08:16:01.025 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-09T08:16:01.031 INFO:teuthology.orchestra.run.smithi133.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-09T08:16:01.147 INFO:teuthology.orchestra.run.smithi133.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 276 kB/s | 80 kB 00:00 2024-05-09T08:16:01.231 INFO:teuthology.orchestra.run.smithi133.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 980 kB/s | 195 kB 00:00 2024-05-09T08:16:01.290 INFO:teuthology.orchestra.run.smithi133.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-05-09T08:16:01.304 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:16:01.305 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:16:01.316 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:16:01.316 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:16:01.317 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:16:01.319 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:16:01.319 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:16:01.322 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:16:01.322 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:16:01.382 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-09T08:16:01.398 INFO:teuthology.orchestra.run.smithi133.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:16:01.407 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:16:01.408 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:16:01.409 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-05-09T08:16:01.410 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-09T08:16:01.411 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:16:01.412 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:16:01.413 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:16:01.414 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:16:01.415 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:16:01.416 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:16:01.417 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:16:01.418 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:16:01.419 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-09T08:16:01.420 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T08:16:01.421 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-05-09T08:16:01.421 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 8 Packages 2024-05-09T08:16:01.421 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:16:01.421 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 225 M 2024-05-09T08:16:01.421 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-09T08:16:01.423 INFO:teuthology.orchestra.run.smithi133.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 803 kB/s | 106 kB 00:00 2024-05-09T08:16:01.465 INFO:teuthology.orchestra.run.smithi133.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-09T08:16:01.508 INFO:teuthology.orchestra.run.smithi133.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-09T08:16:01.583 INFO:teuthology.orchestra.run.smithi133.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-05-09T08:16:01.725 INFO:teuthology.orchestra.run.smithi133.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-05-09T08:16:02.296 INFO:teuthology.orchestra.run.smithi188.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-09T08:16:02.472 INFO:teuthology.orchestra.run.smithi188.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 5.0 MB/s | 889 kB 00:00 2024-05-09T08:16:02.568 INFO:teuthology.orchestra.run.smithi188.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.5 MB/s | 145 kB 00:00 2024-05-09T08:16:02.600 INFO:teuthology.orchestra.run.smithi133.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 122 kB/s | 107 kB 00:00 2024-05-09T08:16:02.650 INFO:teuthology.orchestra.run.smithi133.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 604 kB/s | 689 kB 00:01 2024-05-09T08:16:02.700 INFO:teuthology.orchestra.run.smithi133.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 844 kB/s | 943 kB 00:01 2024-05-09T08:16:02.816 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-09T08:16:02.816 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-09T08:16:02.822 INFO:teuthology.orchestra.run.smithi188.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 9.3 MB/s | 2.4 MB 00:00 2024-05-09T08:16:03.014 INFO:teuthology.orchestra.run.smithi188.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-05-09T08:16:03.123 INFO:teuthology.orchestra.run.smithi188.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-05-09T08:16:03.284 INFO:teuthology.orchestra.run.smithi133.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 31 kB/s | 21 kB 00:00 2024-05-09T08:16:03.468 INFO:teuthology.orchestra.run.smithi133.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 43 kB/s | 33 kB 00:00 2024-05-09T08:16:03.540 INFO:teuthology.orchestra.run.smithi188.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 8.9 MB/s | 4.7 MB 00:00 2024-05-09T08:16:03.626 INFO:teuthology.orchestra.run.smithi133.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 322 kB/s | 313 kB 00:00 2024-05-09T08:16:04.093 INFO:teuthology.orchestra.run.smithi133.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 29 kB/s | 18 kB 00:00 2024-05-09T08:16:04.227 INFO:teuthology.orchestra.run.smithi133.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-05-09T08:16:04.585 INFO:teuthology.orchestra.run.smithi133.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.3 MB/s | 3.0 MB 00:01 2024-05-09T08:16:04.715 INFO:teuthology.orchestra.run.smithi188.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 8.1 MB/s | 21 MB 00:02 2024-05-09T08:16:04.824 INFO:teuthology.orchestra.run.smithi188.stdout:(9/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 8.0 MB/s | 10 MB 00:01 2024-05-09T08:16:04.849 INFO:teuthology.orchestra.run.smithi188.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 188 kB/s | 25 kB 00:00 2024-05-09T08:16:04.940 INFO:teuthology.orchestra.run.smithi188.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 2.4 MB/s | 222 kB 00:00 2024-05-09T08:16:04.944 INFO:teuthology.orchestra.run.smithi133.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 63 kB/s | 45 kB 00:00 2024-05-09T08:16:05.020 INFO:teuthology.orchestra.run.smithi133.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 173 kB/s | 160 kB 00:00 2024-05-09T08:16:05.141 INFO:teuthology.orchestra.run.smithi188.stdout:(12/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 8.3 MB/s | 17 MB 00:02 2024-05-09T08:16:05.166 INFO:teuthology.orchestra.run.smithi188.stdout:(13/156): cephfs-shell-19.0.0-2515.g96805dcf.el 120 kB/s | 27 kB 00:00 2024-05-09T08:16:05.216 INFO:teuthology.orchestra.run.smithi188.stdout:(14/156): libcephfs-devel-19.0.0-2515.g96805dcf 423 kB/s | 32 kB 00:00 2024-05-09T08:16:05.292 INFO:teuthology.orchestra.run.smithi188.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.1 MB/s | 163 kB 00:00 2024-05-09T08:16:05.317 INFO:teuthology.orchestra.run.smithi188.stdout:(16/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 4.8 MB/s | 746 kB 00:00 2024-05-09T08:16:05.401 INFO:teuthology.orchestra.run.smithi188.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 1.1 MB/s | 127 kB 00:00 2024-05-09T08:16:05.459 INFO:teuthology.orchestra.run.smithi188.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 3.4 MB/s | 500 kB 00:00 2024-05-09T08:16:05.543 INFO:teuthology.orchestra.run.smithi188.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 540 kB/s | 45 kB 00:00 2024-05-09T08:16:05.619 INFO:teuthology.orchestra.run.smithi188.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.8 MB/s | 135 kB 00:00 2024-05-09T08:16:05.645 INFO:teuthology.orchestra.run.smithi133.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 41 kB/s | 25 kB 00:00 2024-05-09T08:16:05.686 INFO:teuthology.orchestra.run.smithi133.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 602 kB/s | 662 kB 00:01 2024-05-09T08:16:05.710 INFO:teuthology.orchestra.run.smithi188.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.7 MB/s | 162 kB 00:00 2024-05-09T08:16:05.802 INFO:teuthology.orchestra.run.smithi188.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.4 MB/s | 321 kB 00:00 2024-05-09T08:16:05.936 INFO:teuthology.orchestra.run.smithi188.stdout:(23/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 9.9 MB/s | 5.3 MB 00:00 2024-05-09T08:16:05.961 INFO:teuthology.orchestra.run.smithi188.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 1.8 MB/s | 298 kB 00:00 2024-05-09T08:16:06.012 INFO:teuthology.orchestra.run.smithi133.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 546 kB/s | 582 kB 00:01 2024-05-09T08:16:06.012 INFO:teuthology.orchestra.run.smithi188.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.3 MB/s | 99 kB 00:00 2024-05-09T08:16:06.037 INFO:teuthology.orchestra.run.smithi188.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-09T08:16:06.137 INFO:teuthology.orchestra.run.smithi188.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.7 MB/s | 171 kB 00:00 2024-05-09T08:16:06.221 INFO:teuthology.orchestra.run.smithi188.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 380 kB/s | 32 kB 00:00 2024-05-09T08:16:06.313 INFO:teuthology.orchestra.run.smithi188.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.5 MB/s | 142 kB 00:00 2024-05-09T08:16:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-05-09T08:16:06.372 INFO:teuthology.orchestra.run.smithi188.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 8.6 MB/s | 3.1 MB 00:00 2024-05-09T08:16:06.687 INFO:teuthology.orchestra.run.smithi133.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 293 kB/s | 292 kB 00:00 2024-05-09T08:16:06.922 INFO:teuthology.orchestra.run.smithi188.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.8 MB/s | 3.5 MB 00:00 2024-05-09T08:16:06.929 INFO:teuthology.orchestra.run.smithi133.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 206 kB/s | 188 kB 00:00 2024-05-09T08:16:07.021 INFO:teuthology.orchestra.run.smithi133.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 52 kB/s | 35 kB 00:00 2024-05-09T08:16:07.022 INFO:teuthology.orchestra.run.smithi188.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T08:16:07.181 INFO:teuthology.orchestra.run.smithi188.stdout:(33/156): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-05-09T08:16:07.206 INFO:teuthology.orchestra.run.smithi188.stdout:(34/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 268 kB/s | 49 kB 00:00 2024-05-09T08:16:07.256 INFO:teuthology.orchestra.run.smithi188.stdout:(35/156): ceph-prometheus-alerts-19.0.0-2515.g9 219 kB/s | 16 kB 00:00 2024-05-09T08:16:07.282 INFO:teuthology.orchestra.run.smithi188.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.4 MB/s | 264 kB 00:00 2024-05-09T08:16:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 7.4 MB/s | 763 kB 00:00 2024-05-09T08:16:07.382 INFO:teuthology.orchestra.run.smithi188.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 182 kB/s | 18 kB 00:00 2024-05-09T08:16:08.055 INFO:teuthology.orchestra.run.smithi133.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-05-09T08:16:08.083 INFO:teuthology.orchestra.run.smithi188.stdout:(39/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 15 MB/s | 48 MB 00:03 2024-05-09T08:16:08.155 INFO:teuthology.orchestra.run.smithi133.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-09T08:16:08.322 INFO:teuthology.orchestra.run.smithi133.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-05-09T08:16:08.441 INFO:teuthology.orchestra.run.smithi188.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 74 kB/s | 80 kB 00:01 2024-05-09T08:16:08.841 INFO:teuthology.orchestra.run.smithi188.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 550 kB/s | 803 kB 00:01 2024-05-09T08:16:08.875 INFO:teuthology.orchestra.run.smithi188.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 247 kB/s | 195 kB 00:00 2024-05-09T08:16:09.045 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-09T08:16:09.114 INFO:teuthology.orchestra.run.smithi133.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 139 kB/s | 133 kB 00:00 2024-05-09T08:16:09.139 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:16:09.155 INFO:teuthology.orchestra.run.smithi133.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 829 kB/s | 911 kB 00:01 2024-05-09T08:16:09.177 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:16:09.230 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:16:09.264 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:16:09.290 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:16:09.333 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:16:09.342 INFO:teuthology.orchestra.run.smithi188.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 281 kB/s | 253 kB 00:00 2024-05-09T08:16:09.617 INFO:teuthology.orchestra.run.smithi188.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 144 kB/s | 106 kB 00:00 2024-05-09T08:16:09.652 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:16:09.766 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:09.879 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:09.889 INFO:teuthology.orchestra.run.smithi133.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 3.9 MB/s | 5.8 MB 00:01 2024-05-09T08:16:09.908 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:09.926 INFO:teuthology.orchestra.run.smithi188.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-09T08:16:09.947 INFO:teuthology.orchestra.run.smithi133.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 111 kB/s | 87 kB 00:00 2024-05-09T08:16:09.982 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:10.013 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:16:10.135 INFO:teuthology.orchestra.run.smithi188.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 171 kB/s | 135 kB 00:00 2024-05-09T08:16:10.161 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:16:10.226 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:16:10.254 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:10.264 INFO:teuthology.orchestra.run.smithi133.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 899 kB/s | 1.0 MB 00:01 2024-05-09T08:16:10.365 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:10.385 INFO:teuthology.orchestra.run.smithi188.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-05-09T08:16:10.389 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:10.451 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:10.473 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:10.491 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:10.682 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:10.810 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:10.876 INFO:teuthology.orchestra.run.smithi188.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 226 kB/s | 215 kB 00:00 2024-05-09T08:16:10.889 INFO:teuthology.orchestra.run.smithi133.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 260 kB/s | 244 kB 00:00 2024-05-09T08:16:10.963 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:11.014 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:11.118 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:11.216 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:11.389 INFO:teuthology.orchestra.run.smithi133.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 221 kB/s | 248 kB 00:01 2024-05-09T08:16:11.393 INFO:teuthology.orchestra.run.smithi188.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 547 kB/s | 689 kB 00:01 2024-05-09T08:16:11.429 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:11.541 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:16:11.635 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:16:11.663 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:16:11.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:16:11.715 INFO:teuthology.orchestra.run.smithi133.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-05-09T08:16:11.772 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:16:11.814 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:16:11.835 INFO:teuthology.orchestra.run.smithi188.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 663 kB/s | 943 kB 00:01 2024-05-09T08:16:11.858 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:16:11.890 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:16:11.920 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:16:11.952 INFO:teuthology.orchestra.run.smithi188.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 99 kB/s | 107 kB 00:01 2024-05-09T08:16:11.987 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:16:12.040 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:16:12.070 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:16:12.114 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:16:12.149 INFO:teuthology.orchestra.run.smithi133.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 25 kB/s | 19 kB 00:00 2024-05-09T08:16:12.191 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:16:12.225 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:16:12.268 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:16:12.274 INFO:teuthology.orchestra.run.smithi133.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 19 kB/s | 26 kB 00:01 2024-05-09T08:16:12.297 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:16:12.367 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:16:12.412 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:16:12.486 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:16:12.536 INFO:teuthology.orchestra.run.smithi188.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-05-09T08:16:12.621 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:16:12.673 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:16:12.728 INFO:teuthology.orchestra.run.smithi188.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 352 kB/s | 313 kB 00:00 2024-05-09T08:16:12.741 INFO:teuthology.orchestra.run.smithi133.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 180 kB/s | 172 kB 00:00 2024-05-09T08:16:12.883 INFO:teuthology.orchestra.run.smithi133.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 47 kB/s | 35 kB 00:00 2024-05-09T08:16:13.003 INFO:teuthology.orchestra.run.smithi188.stdout:(54/156): elfutils-libelf-devel-0.190-2.el9.x86 13 kB/s | 21 kB 00:01 2024-05-09T08:16:13.201 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:16:13.278 INFO:teuthology.orchestra.run.smithi188.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 18 kB 00:00 2024-05-09T08:16:13.285 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:16:13.314 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:16:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:16:13.394 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:16:13.445 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:16:13.512 INFO:teuthology.orchestra.run.smithi188.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-09T08:16:13.588 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:16:13.631 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:16:13.686 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:16:13.691 INFO:teuthology.orchestra.run.smithi133.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-05-09T08:16:13.720 INFO:teuthology.orchestra.run.smithi188.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-05-09T08:16:13.725 INFO:teuthology.orchestra.run.smithi133.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 255 kB/s | 250 kB 00:00 2024-05-09T08:16:13.759 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:16:13.766 INFO:teuthology.orchestra.run.smithi133.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 252 kB/s | 222 kB 00:00 2024-05-09T08:16:13.790 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:16:13.826 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:16:13.873 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:16:13.917 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:16:13.961 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:16:14.002 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:16:14.087 INFO:teuthology.orchestra.run.smithi188.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 198 kB/s | 160 kB 00:00 2024-05-09T08:16:14.139 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:16:14.183 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:16:14.187 INFO:teuthology.orchestra.run.smithi188.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 67 kB/s | 45 kB 00:00 2024-05-09T08:16:14.213 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:16:14.255 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:16:14.350 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:16:14.459 INFO:teuthology.orchestra.run.smithi133.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 58 kB/s | 42 kB 00:00 2024-05-09T08:16:14.460 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:16:14.492 INFO:teuthology.orchestra.run.smithi133.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 97 kB/s | 77 kB 00:00 2024-05-09T08:16:14.613 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:16:14.654 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:16:14.664 INFO:teuthology.orchestra.run.smithi188.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 703 kB/s | 662 kB 00:00 2024-05-09T08:16:14.676 INFO:teuthology.orchestra.run.smithi133.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 297 kB/s | 269 kB 00:00 2024-05-09T08:16:14.679 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:16:14.761 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:16:14.763 INFO:teuthology.orchestra.run.smithi188.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 44 kB/s | 25 kB 00:00 2024-05-09T08:16:14.891 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:16:14.928 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:16:14.956 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:16:14.990 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:16:15.029 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:16:15.055 INFO:teuthology.orchestra.run.smithi188.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 602 kB/s | 582 kB 00:00 2024-05-09T08:16:15.069 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:16:15.098 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:16:15.133 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:16:15.167 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:16:15.200 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:16:15.234 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:16:15.321 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:16:15.330 INFO:teuthology.orchestra.run.smithi188.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-05-09T08:16:15.334 INFO:teuthology.orchestra.run.smithi133.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 182 kB/s | 159 kB 00:00 2024-05-09T08:16:15.365 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:16:15.396 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:16:15.435 INFO:teuthology.orchestra.run.smithi133.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 296 kB/s | 279 kB 00:00 2024-05-09T08:16:15.443 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:16:15.460 INFO:teuthology.orchestra.run.smithi133.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 65 kB/s | 51 kB 00:00 2024-05-09T08:16:15.542 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:16:15.589 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:16:15.680 INFO:teuthology.orchestra.run.smithi188.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 319 kB/s | 292 kB 00:00 2024-05-09T08:16:15.889 INFO:teuthology.orchestra.run.smithi188.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 226 kB/s | 188 kB 00:00 2024-05-09T08:16:15.964 INFO:teuthology.orchestra.run.smithi188.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 58 kB/s | 35 kB 00:00 2024-05-09T08:16:16.118 INFO:teuthology.orchestra.run.smithi133.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 69 kB/s | 54 kB 00:00 2024-05-09T08:16:16.235 INFO:teuthology.orchestra.run.smithi133.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 54 kB/s | 42 kB 00:00 2024-05-09T08:16:16.317 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:16:16.361 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:16:16.395 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:16:16.493 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:16:16.545 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:16:16.632 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:16:16.672 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:16:16.850 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:16:16.886 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:16:16.961 INFO:teuthology.orchestra.run.smithi133.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 52 kB/s | 44 kB 00:00 2024-05-09T08:16:16.964 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:16:16.981 INFO:teuthology.orchestra.run.smithi188.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-05-09T08:16:17.106 INFO:teuthology.orchestra.run.smithi188.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-09T08:16:17.173 INFO:teuthology.orchestra.run.smithi188.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-05-09T08:16:17.186 INFO:teuthology.orchestra.run.smithi133.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.3 MB/s | 16 MB 00:01 2024-05-09T08:16:17.219 INFO:teuthology.orchestra.run.smithi133.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 225 kB/s | 221 kB 00:00 2024-05-09T08:16:17.616 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:16:17.735 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:16:17.865 INFO:teuthology.orchestra.run.smithi188.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 176 kB/s | 133 kB 00:00 2024-05-09T08:16:17.903 INFO:teuthology.orchestra.run.smithi133.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 38 kB/s | 36 kB 00:00 2024-05-09T08:16:17.961 INFO:teuthology.orchestra.run.smithi133.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 61 kB/s | 47 kB 00:00 2024-05-09T08:16:17.999 INFO:teuthology.orchestra.run.smithi188.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 896 kB/s | 911 kB 00:01 2024-05-09T08:16:18.279 INFO:teuthology.orchestra.run.smithi133.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 288 kB/s | 305 kB 00:01 2024-05-09T08:16:18.424 INFO:teuthology.orchestra.run.smithi188.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.6 MB/s | 5.8 MB 00:01 2024-05-09T08:16:18.613 INFO:teuthology.orchestra.run.smithi133.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 82 kB/s | 58 kB 00:00 2024-05-09T08:16:18.724 INFO:teuthology.orchestra.run.smithi188.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 121 kB/s | 87 kB 00:00 2024-05-09T08:16:18.755 INFO:teuthology.orchestra.run.smithi133.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 118 kB/s | 94 kB 00:00 2024-05-09T08:16:18.804 INFO:teuthology.orchestra.run.smithi133.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2024-05-09T08:16:18.894 INFO:teuthology.orchestra.run.smithi188.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-09T08:16:18.971 INFO:teuthology.orchestra.run.smithi133.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-05-09T08:16:18.972 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:16:19.046 INFO:teuthology.orchestra.run.smithi133.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 83 kB/s | 64 kB 00:00 2024-05-09T08:16:19.056 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:16:19.084 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:16:19.117 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:16:19.372 INFO:teuthology.orchestra.run.smithi133.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 59 kB/s | 45 kB 00:00 2024-05-09T08:16:19.429 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:16:19.472 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:19.563 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:19.594 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:16:19.629 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:16:19.705 INFO:teuthology.orchestra.run.smithi133.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 526 kB/s | 309 kB 00:00 2024-05-09T08:16:19.741 INFO:teuthology.orchestra.run.smithi188.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 240 kB/s | 244 kB 00:01 2024-05-09T08:16:19.833 INFO:teuthology.orchestra.run.smithi188.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-05-09T08:16:19.847 INFO:teuthology.orchestra.run.smithi133.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 626 kB/s | 548 kB 00:00 2024-05-09T08:16:19.872 INFO:teuthology.orchestra.run.smithi133.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 39 kB/s | 19 kB 00:00 2024-05-09T08:16:19.934 INFO:teuthology.orchestra.run.smithi188.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 238 kB/s | 248 kB 00:01 2024-05-09T08:16:20.100 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:16:20.134 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:20.188 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:20.210 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:16:20.542 INFO:teuthology.orchestra.run.smithi188.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 32 kB/s | 26 kB 00:00 2024-05-09T08:16:20.575 INFO:teuthology.orchestra.run.smithi188.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 26 kB/s | 19 kB 00:00 2024-05-09T08:16:20.926 INFO:teuthology.orchestra.run.smithi188.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 174 kB/s | 172 kB 00:00 2024-05-09T08:16:21.193 INFO:teuthology.orchestra.run.smithi188.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 53 kB/s | 35 kB 00:00 2024-05-09T08:16:21.768 INFO:teuthology.orchestra.run.smithi188.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 298 kB/s | 250 kB 00:00 2024-05-09T08:16:21.843 INFO:teuthology.orchestra.run.smithi188.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-05-09T08:16:22.010 INFO:teuthology.orchestra.run.smithi188.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-05-09T08:16:22.033 INFO:teuthology.orchestra.run.smithi133.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-05-09T08:16:22.231 INFO:teuthology.orchestra.run.smithi133.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 11 kB/s | 26 kB 00:02 2024-05-09T08:16:22.256 INFO:teuthology.orchestra.run.smithi133.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 20 kB/s | 49 kB 00:02 2024-05-09T08:16:22.273 INFO:teuthology.orchestra.run.smithi133.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 283 kB/s | 67 kB 00:00 2024-05-09T08:16:22.365 INFO:teuthology.orchestra.run.smithi133.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-09T08:16:22.463 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:22.493 INFO:teuthology.orchestra.run.smithi188.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 106 kB/s | 77 kB 00:00 2024-05-09T08:16:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:22.519 INFO:teuthology.orchestra.run.smithi188.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 63 kB/s | 42 kB 00:00 2024-05-09T08:16:22.665 INFO:teuthology.orchestra.run.smithi133.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.0 MB/s | 839 kB 00:00 2024-05-09T08:16:22.849 INFO:teuthology.orchestra.run.smithi133.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 953 kB/s | 548 kB 00:00 2024-05-09T08:16:22.860 INFO:teuthology.orchestra.run.smithi188.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 327 kB/s | 269 kB 00:00 2024-05-09T08:16:22.874 INFO:teuthology.orchestra.run.smithi133.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 86 kB/s | 43 kB 00:00 2024-05-09T08:16:22.899 INFO:teuthology.orchestra.run.smithi133.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 139 kB/s | 32 kB 00:00 2024-05-09T08:16:22.924 INFO:teuthology.orchestra.run.smithi133.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-05-09T08:16:23.008 INFO:teuthology.orchestra.run.smithi133.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-05-09T08:16:23.038 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:23.109 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:23.117 INFO:teuthology.orchestra.run.smithi133.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-05-09T08:16:23.185 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:23.185 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:16:23.186 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:16:23.186 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:23.213 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:16:23.217 INFO:teuthology.orchestra.run.smithi133.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-05-09T08:16:23.242 INFO:teuthology.orchestra.run.smithi133.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 152 kB/s | 35 kB 00:00 2024-05-09T08:16:23.252 INFO:teuthology.orchestra.run.smithi188.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 209 kB/s | 159 kB 00:00 2024-05-09T08:16:23.326 INFO:teuthology.orchestra.run.smithi133.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 987 kB/s | 206 kB 00:00 2024-05-09T08:16:23.403 INFO:teuthology.orchestra.run.smithi188.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 315 kB/s | 279 kB 00:00 2024-05-09T08:16:23.553 INFO:teuthology.orchestra.run.smithi188.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 74 kB/s | 51 kB 00:00 2024-05-09T08:16:23.928 INFO:teuthology.orchestra.run.smithi188.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-05-09T08:16:23.959 INFO:teuthology.orchestra.run.smithi133.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-05-09T08:16:23.993 INFO:teuthology.orchestra.run.smithi133.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 193 kB/s | 144 kB 00:00 2024-05-09T08:16:24.057 INFO:teuthology.orchestra.run.smithi133.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 15 kB/s | 11 kB 00:00 2024-05-09T08:16:24.085 INFO:teuthology.orchestra.run.smithi133.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 143 kB/s | 18 kB 00:00 2024-05-09T08:16:24.119 INFO:teuthology.orchestra.run.smithi133.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 386 kB/s | 23 kB 00:00 2024-05-09T08:16:24.144 INFO:teuthology.orchestra.run.smithi133.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 229 kB/s | 19 kB 00:00 2024-05-09T08:16:24.169 INFO:teuthology.orchestra.run.smithi133.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-05-09T08:16:24.203 INFO:teuthology.orchestra.run.smithi188.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-05-09T08:16:24.220 INFO:teuthology.orchestra.run.smithi133.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 628 kB/s | 46 kB 00:00 2024-05-09T08:16:24.495 INFO:teuthology.orchestra.run.smithi133.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-05-09T08:16:24.520 INFO:teuthology.orchestra.run.smithi133.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-05-09T08:16:24.545 INFO:teuthology.orchestra.run.smithi133.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 178 kB/s | 58 kB 00:00 2024-05-09T08:16:24.554 INFO:teuthology.orchestra.run.smithi188.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-05-09T08:16:24.612 INFO:teuthology.orchestra.run.smithi133.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-09T08:16:24.637 INFO:teuthology.orchestra.run.smithi133.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 183 kB/s | 16 kB 00:00 2024-05-09T08:16:24.663 INFO:teuthology.orchestra.run.smithi133.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 784 kB/s | 90 kB 00:00 2024-05-09T08:16:24.688 INFO:teuthology.orchestra.run.smithi133.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-05-09T08:16:24.713 INFO:teuthology.orchestra.run.smithi133.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 415 kB/s | 31 kB 00:00 2024-05-09T08:16:24.780 INFO:teuthology.orchestra.run.smithi133.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-05-09T08:16:24.805 INFO:teuthology.orchestra.run.smithi133.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-05-09T08:16:24.830 INFO:teuthology.orchestra.run.smithi133.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 310 kB/s | 36 kB 00:00 2024-05-09T08:16:24.856 INFO:teuthology.orchestra.run.smithi133.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 688 kB/s | 51 kB 00:00 2024-05-09T08:16:24.948 INFO:teuthology.orchestra.run.smithi133.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-09T08:16:24.954 INFO:teuthology.orchestra.run.smithi188.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-09T08:16:24.981 INFO:teuthology.orchestra.run.smithi133.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 603 kB/s | 90 kB 00:00 2024-05-09T08:16:25.073 INFO:teuthology.orchestra.run.smithi133.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-05-09T08:16:25.090 INFO:teuthology.orchestra.run.smithi133.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-05-09T08:16:25.149 INFO:teuthology.orchestra.run.smithi133.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-05-09T08:16:25.171 INFO:teuthology.orchestra.run.smithi188.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 228 kB/s | 221 kB 00:00 2024-05-09T08:16:25.238 INFO:teuthology.orchestra.run.smithi188.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 53 kB/s | 36 kB 00:00 2024-05-09T08:16:25.482 INFO:teuthology.orchestra.run.smithi133.stdout:(148/156): librados2-19.0.0-2515.g96805dcf.el9. 8.7 MB/s | 3.4 MB 00:00 2024-05-09T08:16:25.566 INFO:teuthology.orchestra.run.smithi133.stdout:(149/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-05-09T08:16:25.571 INFO:teuthology.orchestra.run.smithi188.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-05-09T08:16:25.650 INFO:teuthology.orchestra.run.smithi133.stdout:(150/156): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-05-09T08:16:25.775 INFO:teuthology.orchestra.run.smithi133.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 336 kB/s | 42 kB 00:00 2024-05-09T08:16:25.909 INFO:teuthology.orchestra.run.smithi133.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.4 MB/s | 2.3 MB 00:00 2024-05-09T08:16:25.913 INFO:teuthology.orchestra.run.smithi188.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-05-09T08:16:25.976 INFO:teuthology.orchestra.run.smithi133.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.2 MB/s | 248 kB 00:00 2024-05-09T08:16:26.030 INFO:teuthology.orchestra.run.smithi188.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-05-09T08:16:26.126 INFO:teuthology.orchestra.run.smithi133.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 1.3 MB 00:00 2024-05-09T08:16:26.176 INFO:teuthology.orchestra.run.smithi133.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.0 MB/s | 1.8 MB 00:00 2024-05-09T08:16:26.298 INFO:teuthology.orchestra.run.smithi188.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 129 kB/s | 94 kB 00:00 2024-05-09T08:16:26.456 INFO:teuthology.orchestra.run.smithi188.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-05-09T08:16:26.559 INFO:teuthology.orchestra.run.smithi133.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.1 MB/s | 2.4 MB 00:00 2024-05-09T08:16:26.564 INFO:teuthology.orchestra.run.smithi188.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 98 kB/s | 64 kB 00:00 2024-05-09T08:16:26.571 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:16:26.571 INFO:teuthology.orchestra.run.smithi133.stdout:Total 6.9 MB/s | 225 MB 00:32 2024-05-09T08:16:26.640 INFO:teuthology.orchestra.run.smithi188.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-05-09T08:16:26.682 INFO:teuthology.orchestra.run.smithi188.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-05-09T08:16:26.799 INFO:teuthology.orchestra.run.smithi188.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-05-09T08:16:26.824 INFO:teuthology.orchestra.run.smithi188.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-05-09T08:16:26.841 INFO:teuthology.orchestra.run.smithi188.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-05-09T08:16:26.958 INFO:teuthology.orchestra.run.smithi188.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-05-09T08:16:26.975 INFO:teuthology.orchestra.run.smithi188.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 196 kB/s | 26 kB 00:00 2024-05-09T08:16:26.991 INFO:teuthology.orchestra.run.smithi188.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-05-09T08:16:27.017 INFO:teuthology.orchestra.run.smithi188.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T08:16:27.042 INFO:teuthology.orchestra.run.smithi188.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-09T08:16:27.084 INFO:teuthology.orchestra.run.smithi188.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-09T08:16:27.143 INFO:teuthology.orchestra.run.smithi188.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-09T08:16:27.168 INFO:teuthology.orchestra.run.smithi188.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 350 kB/s | 43 kB 00:00 2024-05-09T08:16:27.193 INFO:teuthology.orchestra.run.smithi188.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-05-09T08:16:27.219 INFO:teuthology.orchestra.run.smithi188.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-05-09T08:16:27.244 INFO:teuthology.orchestra.run.smithi188.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-09T08:16:27.269 INFO:teuthology.orchestra.run.smithi188.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-09T08:16:27.294 INFO:teuthology.orchestra.run.smithi188.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-05-09T08:16:27.320 INFO:teuthology.orchestra.run.smithi188.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-05-09T08:16:27.345 INFO:teuthology.orchestra.run.smithi188.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T08:16:27.412 INFO:teuthology.orchestra.run.smithi188.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-05-09T08:16:27.437 INFO:teuthology.orchestra.run.smithi188.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-05-09T08:16:27.471 INFO:teuthology.orchestra.run.smithi188.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-05-09T08:16:27.496 INFO:teuthology.orchestra.run.smithi188.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 210 kB/s | 18 kB 00:00 2024-05-09T08:16:27.522 INFO:teuthology.orchestra.run.smithi188.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-05-09T08:16:27.538 INFO:teuthology.orchestra.run.smithi188.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 289 kB/s | 19 kB 00:00 2024-05-09T08:16:27.564 INFO:teuthology.orchestra.run.smithi188.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 292 kB/s | 20 kB 00:00 2024-05-09T08:16:27.597 INFO:teuthology.orchestra.run.smithi188.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-05-09T08:16:27.623 INFO:teuthology.orchestra.run.smithi188.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-05-09T08:16:27.648 INFO:teuthology.orchestra.run.smithi188.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 948 kB/s | 79 kB 00:00 2024-05-09T08:16:27.673 INFO:teuthology.orchestra.run.smithi188.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-09T08:16:27.690 INFO:teuthology.orchestra.run.smithi188.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-05-09T08:16:27.716 INFO:teuthology.orchestra.run.smithi188.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-09T08:16:27.740 INFO:teuthology.orchestra.run.smithi188.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T08:16:27.757 INFO:teuthology.orchestra.run.smithi188.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-05-09T08:16:27.774 INFO:teuthology.orchestra.run.smithi188.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-05-09T08:16:27.800 INFO:teuthology.orchestra.run.smithi188.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-09T08:16:27.825 INFO:teuthology.orchestra.run.smithi188.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-05-09T08:16:27.842 INFO:teuthology.orchestra.run.smithi188.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 531 kB/s | 36 kB 00:00 2024-05-09T08:16:27.859 INFO:teuthology.orchestra.run.smithi188.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 870 kB/s | 51 kB 00:00 2024-05-09T08:16:27.892 INFO:teuthology.orchestra.run.smithi188.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-09T08:16:27.918 INFO:teuthology.orchestra.run.smithi188.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T08:16:27.943 INFO:teuthology.orchestra.run.smithi188.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T08:16:27.968 INFO:teuthology.orchestra.run.smithi188.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-05-09T08:16:28.018 INFO:teuthology.orchestra.run.smithi188.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-09T08:16:28.044 INFO:teuthology.orchestra.run.smithi188.stdout:(148/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-09T08:16:28.252 INFO:teuthology.orchestra.run.smithi188.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 12 MB/s | 3.4 MB 00:00 2024-05-09T08:16:28.519 INFO:teuthology.orchestra.run.smithi188.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-09T08:16:29.111 INFO:teuthology.orchestra.run.smithi188.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-05-09T08:16:29.378 INFO:teuthology.orchestra.run.smithi188.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 50 kB/s | 42 kB 00:00 2024-05-09T08:16:29.569 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-05-09T08:16:29.604 INFO:teuthology.orchestra.run.smithi188.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2024-05-09T08:16:29.811 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-05-09T08:16:29.811 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-05-09T08:16:30.179 INFO:teuthology.orchestra.run.smithi188.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 233 kB/s | 248 kB 00:01 2024-05-09T08:16:30.404 INFO:teuthology.orchestra.run.smithi188.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T08:16:30.796 INFO:teuthology.orchestra.run.smithi188.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-05-09T08:16:30.801 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:16:30.801 INFO:teuthology.orchestra.run.smithi188.stdout:Total 7.6 MB/s | 225 MB 00:29 2024-05-09T08:16:32.005 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-09T08:16:32.160 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-09T08:16:32.160 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-09T08:16:32.464 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-05-09T08:16:32.465 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-05-09T08:16:34.027 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-09T08:16:34.028 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-09T08:16:34.604 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:16:34.604 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-09T08:16:34.604 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-09T08:16:34.604 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-09T08:16:34.604 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-09T08:16:34.605 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-09T08:16:34.605 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-05-09T08:16:34.605 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-09T08:16:34.605 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:34.774 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:16:34.790 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:16:34.790 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:34.791 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:16:34.791 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:36.278 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:16:36.304 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:16:36.304 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:36.304 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:16:36.304 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:16:36.304 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:16:36.305 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:36.528 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:16:36.551 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:16:36.551 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:36.551 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:16:36.552 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:16:36.552 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:16:36.552 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:36.987 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:16:37.010 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:16:37.010 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:37.010 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:16:37.011 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:16:37.011 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:16:37.011 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:37.126 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:16:37.183 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:16:37.239 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:16:37.425 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:16:37.480 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:16:37.558 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:16:37.647 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:16:37.788 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:16:37.838 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:16:37.870 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:16:37.897 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:16:38.254 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:16:38.279 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:16:38.310 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:16:38.383 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:16:38.419 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:16:38.771 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-05-09T08:16:38.857 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:16:38.906 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:16:38.992 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:16:39.039 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:16:39.075 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:16:39.171 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:16:39.458 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:16:39.495 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:16:39.595 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:16:39.691 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:39.864 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:39.891 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:40.017 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:40.066 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:16:40.089 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-09T08:16:40.183 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:16:40.221 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:16:40.275 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:16:40.309 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:16:40.334 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:16:40.360 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:16:40.385 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:16:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:16:40.445 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:40.464 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:16:40.488 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:16:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:40.590 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:16:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:40.690 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:40.703 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:16:40.723 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:40.752 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:16:40.783 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:40.789 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:16:40.810 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:16:40.813 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:40.874 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:16:40.885 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:16:40.931 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:40.962 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:40.982 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:16:41.030 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:16:41.053 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:41.057 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:41.083 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:16:41.230 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:16:41.279 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:16:41.308 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:41.366 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:41.411 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:41.433 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:41.496 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:16:41.518 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:41.537 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:41.542 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:16:41.625 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:41.693 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:41.815 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:41.823 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:16:41.899 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:41.922 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:16:41.959 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:16:41.982 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:16:41.982 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:41.982 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:16:41.982 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:16:41.983 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:16:41.983 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:41.985 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:42.026 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:16:42.043 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:16:42.049 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:16:42.049 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:42.049 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:16:42.049 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:42.094 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:16:42.116 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:16:42.116 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:42.117 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:16:42.117 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:16:42.117 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:16:42.117 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:42.145 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:42.211 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:42.242 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:42.405 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:16:42.419 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:16:42.433 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:16:42.442 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:16:42.442 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:16:42.443 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:16:42.443 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:16:42.443 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:16:42.443 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:42.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:16:42.557 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:16:42.597 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:16:42.601 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:16:42.642 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:16:42.644 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:16:42.685 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:16:42.748 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:16:42.754 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:16:42.784 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:16:42.799 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:16:42.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:16:42.860 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:16:42.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:16:42.888 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:16:42.912 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:16:42.945 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:16:42.955 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:16:43.010 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:16:43.050 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:16:43.058 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:16:43.085 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:16:43.131 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:16:43.164 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:16:43.168 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:16:43.204 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:16:43.216 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:16:43.247 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:16:43.276 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:16:43.325 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:16:43.345 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:16:43.366 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:16:43.391 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:16:43.412 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:16:43.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:16:43.464 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:16:43.563 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:16:43.620 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:16:43.621 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:16:43.676 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:16:43.727 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:16:43.900 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:16:43.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:16:44.190 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:16:44.280 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:16:44.310 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:16:44.353 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:16:44.390 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:16:44.432 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:16:44.636 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:16:44.644 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:16:44.710 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:16:44.734 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:16:44.771 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:16:44.773 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:16:44.816 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:16:44.837 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:16:44.865 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:16:44.869 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:16:44.906 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:16:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:16:44.952 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:16:44.989 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:16:45.031 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:16:45.073 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:16:45.118 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:16:45.189 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:16:45.207 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:16:45.254 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:16:45.268 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:16:45.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:16:45.326 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:16:45.360 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:16:45.407 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:16:45.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:16:45.446 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:16:45.452 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:16:45.492 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:16:45.514 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:16:45.517 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:16:45.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:16:45.555 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:16:45.615 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:16:45.617 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:16:45.674 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:16:45.736 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:16:45.775 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:16:45.802 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:16:45.853 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:16:45.854 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:16:45.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:16:45.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:16:45.924 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:16:45.938 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:16:45.954 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:16:45.983 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:16:45.988 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:16:46.022 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:16:46.063 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:16:46.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:16:46.108 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:16:46.147 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:16:46.193 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:16:46.193 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:16:46.229 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:16:46.237 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:16:46.262 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:16:46.268 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:16:46.314 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:16:46.413 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:16:46.451 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:16:46.461 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:16:46.623 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:16:46.660 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:16:46.697 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:16:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:16:46.807 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:16:46.863 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:16:46.905 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:16:46.940 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:16:46.952 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:16:46.989 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:16:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:16:47.030 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:16:47.062 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:16:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:16:47.089 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:16:47.125 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:16:47.182 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:16:47.225 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:16:47.259 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:16:47.265 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:16:47.278 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:16:47.318 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:16:47.331 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:16:47.349 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:16:47.355 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:16:47.373 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:16:47.378 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:16:47.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:16:47.424 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:16:47.444 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:16:47.494 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:16:47.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:16:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:16:47.650 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:16:47.721 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:16:47.759 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:16:47.836 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:16:48.013 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:16:48.047 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:16:48.067 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:16:48.469 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:16:48.529 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:16:48.577 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:16:48.589 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:16:48.610 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:16:48.731 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:16:48.796 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:16:48.892 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:16:48.923 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:16:48.928 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:16:49.009 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:16:49.050 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:16:49.131 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:16:49.158 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:16:49.184 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:16:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:16:49.230 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:16:49.230 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:16:49.284 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:16:49.306 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:16:49.347 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:16:49.347 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:16:49.419 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:16:49.444 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:16:49.449 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:16:49.450 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:16:49.632 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:16:49.632 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:16:49.717 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:16:49.780 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:16:49.793 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:16:49.880 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:16:49.908 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:16:49.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:16:50.116 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:16:50.252 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:16:50.254 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:16:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:50.380 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:50.411 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:16:50.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:16:50.915 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:16:50.942 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:51.014 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:51.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:16:51.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:16:51.973 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:16:52.009 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:16:52.044 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:16:52.436 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:16:52.477 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:52.618 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:16:52.650 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:16:52.706 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:16:53.245 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:16:53.251 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:53.280 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:53.355 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:16:53.373 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:53.382 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:16:53.910 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:53.951 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:54.051 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:54.052 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:16:54.052 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:16:54.052 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:16:54.086 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:16:55.268 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:16:55.269 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:16:55.270 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:16:55.283 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:16:55.284 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:16:55.285 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:16:55.286 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:16:55.287 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:16:55.288 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:16:55.289 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:16:55.290 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:16:55.291 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:16:55.292 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:16:55.993 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:56.037 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:56.685 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:16:56.732 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:56.896 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:16:56.917 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:16:56.917 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:16:56.917 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:16:56.932 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.672 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.673 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.674 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:16:57.675 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:16:57.676 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:16:57.677 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:16:57.678 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:16:57.679 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:16:57.680 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.681 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:16:57.682 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:16:57.683 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:16:57.684 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:16:57.684 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T08:16:57.684 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-09T08:16:58.070 DEBUG:teuthology.parallel:result is None 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-05-09T08:17:05.767 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-05-09T08:17:05.768 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:05.922 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:17:05.938 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:17:05.938 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:05.938 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:17:05.938 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:07.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:17:07.437 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:07.676 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:17:07.699 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:17:07.700 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:07.700 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:17:07.700 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:17:07.700 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:17:07.700 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:08.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:17:08.168 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:08.274 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:17:08.331 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:17:08.395 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:17:08.578 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:17:08.628 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:17:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:17:08.794 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:17:08.934 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:17:08.978 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:17:09.011 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:17:09.038 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:17:09.487 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:17:09.511 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:17:09.543 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:17:09.613 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:17:09.643 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:17:09.671 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:17:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-05-09T08:17:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-05-09T08:17:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-05-09T08:17:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-05-09T08:17:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-05-09T08:17:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-05-09T08:17:09.672 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-05-09T08:17:09.672 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:09.881 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:17:09.924 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:17:09.924 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:09.925 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:17:09.925 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:10.666 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:17:10.703 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:17:11.549 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:17:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:17:11.650 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:11.650 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:17:11.650 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:17:11.651 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:17:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:11.660 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:17:11.687 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:17:11.786 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:17:11.907 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:17:11.948 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:17:11.976 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:11.980 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:17:12.004 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:12.056 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:17:12.068 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:17:12.164 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:17:12.221 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:17:12.466 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:17:12.538 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:17:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:17:12.538 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:17:12.539 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:17:12.539 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:12.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:17:12.729 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:17:12.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:17:13.025 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:17:13.090 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:17:13.124 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:17:13.158 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:17:13.176 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:17:13.181 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:13.225 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:17:13.247 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:17:13.248 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:13.248 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:17:13.248 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:13.283 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:17:13.293 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:17:13.315 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:17:13.315 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:13.315 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:17:13.316 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:17:13.316 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:17:13.316 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:13.465 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:17:13.513 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:17:13.564 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:17:13.595 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:17:13.618 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:17:13.641 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:14.245 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:17:14.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:17:14.344 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:17:14.458 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:17:14.492 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:17:15.813 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:17:15.852 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:17:16.968 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:17:17.002 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:17:17.148 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:17:17.327 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:17:17.355 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:17:17.428 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:17:17.428 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:17.428 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:17:17.428 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:17:17.428 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:17:17.429 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:17.488 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:17:17.528 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:17:17.638 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:17:17.709 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:17:18.113 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:17:18.154 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:17:18.203 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:17:18.236 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:17:18.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:17:18.442 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:17:18.484 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:17:18.515 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:17:18.544 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:17:18.590 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:17:18.699 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:17:18.732 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:17:18.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:18.883 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:17:18.953 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:17:18.953 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:18.954 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:17:18.954 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:19.025 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:17:19.101 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:19.201 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:19.238 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:19.258 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:19.467 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:17:19.538 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:20.067 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:17:20.159 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:17:20.199 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:17:20.280 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:17:20.312 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:17:20.338 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:17:20.339 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:17:20.423 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:17:20.455 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:17:20.456 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:17:20.548 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:17:20.573 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:17:20.579 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:20.579 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:20.767 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:20.767 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:17:20.885 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:17:20.955 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:24.497 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:17:24.551 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:17:24.613 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:17:24.646 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:17:24.685 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:17:24.985 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:17:25.043 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:17:25.081 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:17:25.135 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:17:25.183 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:17:26.140 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:26.196 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:26.251 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:17:26.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:17:26.460 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:17:26.461 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:17:26.474 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:17:26.475 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:17:26.476 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:17:26.477 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:17:26.478 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:17:26.479 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:17:26.480 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:17:26.481 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:17:26.482 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:17:26.483 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:17:26.484 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:17:26.485 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:17:27.601 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:17:27.803 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:17:27.888 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:17:28.061 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:17:28.104 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:17:28.142 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:17:28.142 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:17:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:17:28.316 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:17:28.317 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:17:28.453 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:17:28.493 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:17:28.526 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:28.526 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:28.864 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:17:28.864 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:17:29.000 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.005 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.006 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.007 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:17:29.008 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:17:29.009 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:17:29.010 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:17:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:17:29.012 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:17:29.013 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:17:29.014 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:17:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.016 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:17:29.017 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-09T08:17:29.132 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:29.380 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:29.380 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-09T08:17:29.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:29.429 DEBUG:teuthology.parallel:result is None 2024-05-09T08:17:35.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:17:35.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:17:35.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:17:35.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:17:35.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:17:35.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:17:35.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:17:35.019 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:17:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:17:35.021 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:17:35.022 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:17:35.023 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:17:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:17:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:17:35.026 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:17:35.027 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:17:35.028 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:17:35.029 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:17:37.122 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.123 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.124 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:17:37.125 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.126 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.127 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:17:37.128 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:17:37.129 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:17:37.130 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:17:37.131 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:17:37.132 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:17:37.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:17:37.135 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:17:37.136 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:17:37.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:17:37.136 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-05-09T08:17:37.748 DEBUG:teuthology.parallel:result is None 2024-05-09T08:17:37.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:37.978 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:17:38.027 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.027 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.028 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:17:38.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:38.257 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:17:38.283 INFO:teuthology.orchestra.run.smithi160.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.284 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.284 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:17:38.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:38.510 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:17:38.536 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.536 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:17:38.536 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:17:38.538 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T08:17:38.539 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:17:38.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:17:38.615 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:17:38.615 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:17:38.646 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:17:38.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:17:38.677 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T08:17:38.677 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:17:38.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:17:38.738 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:17:38.839 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:17:38.839 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:17:38.868 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:17:38.937 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:17:38.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:17:38.967 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:17:39.032 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T08:17:39.032 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:17:39.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:17:39.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:17:39.209 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:17:39.209 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:17:39.233 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:17:39.298 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:17:39.298 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:17:39.323 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:17:39.387 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T08:17:39.387 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:17:39.387 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:17:39.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:17:39.558 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:17:39.558 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:17:39.583 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:17:39.647 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:17:39.647 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:17:39.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:17:39.735 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T08:17:39.838 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T08:17:39.839 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:39.839 INFO:tasks.cephadm:Cluster fsid is 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:17:39.839 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T08:17:39.840 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.133', 'mon.b': '172.21.15.160', 'mon.c': '172.21.15.188'} 2024-05-09T08:17:39.840 INFO:tasks.cephadm:First mon is mon.a on smithi133 2024-05-09T08:17:39.840 INFO:tasks.cephadm:First mgr is x 2024-05-09T08:17:39.840 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T08:17:39.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-05-09T08:17:39.911 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2024-05-09T08:17:39.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-05-09T08:17:39.958 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T08:17:39.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:40.182 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55097/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2515-g96805dcf', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2515.g96805dcf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 19:00:25.721799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-09T08:17:40.404 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:17:40.405 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm 2024-05-09T08:17:40.405 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:40.942 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:17 /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:40.943 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:41.586 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:17 /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:41.587 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:42.094 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 782039 May 9 08:17 /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:42.095 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:42.149 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:42.168 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T08:17:42.203 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc on all hosts... 2024-05-09T08:17:42.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:17:42.208 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:17:42.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc pull 2024-05-09T08:17:43.697 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:17:43.889 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:17:44.706 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:18:26.968 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:18:26.969 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:18:33.546 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-05-09T08:18:33.547 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662", 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac" 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2024-05-09T08:18:33.599 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T08:18:33.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-05-09T08:18:33.718 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2024-05-09T08:18:33.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-05-09T08:18:33.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-05-09T08:18:33.867 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2024-05-09T08:18:33.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-05-09T08:18:33.915 INFO:tasks.cephadm:Writing seed config... 2024-05-09T08:18:33.916 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T08:18:33.916 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T08:18:33.916 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T08:18:33.916 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T08:18:33.916 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T08:18:33.917 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T08:18:33.918 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T08:18:33.919 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T08:18:33.920 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T08:18:33.920 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T08:18:33.920 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T08:18:33.920 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-09T08:18:33.921 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:18:33.921 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T08:18:34.048 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 99f031e6-0ddc-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T08:18:34.048 DEBUG:teuthology.orchestra.run.smithi133:mon.a> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service 2024-05-09T08:18:34.094 DEBUG:teuthology.orchestra.run.smithi133:mgr.x> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x.service 2024-05-09T08:18:34.140 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T08:18:34.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc -v bootstrap --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.133 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:18:34.408 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:18:34.408 INFO:teuthology.orchestra.run.smithi133.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', '-v', 'bootstrap', '--fsid', '99f031e6-0ddc-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.133', '--skip-admin-label'] 2024-05-09T08:18:34.408 INFO:teuthology.orchestra.run.smithi133.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T08:18:34.408 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying podman|docker is present... 2024-05-09T08:18:34.461 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T08:18:34.462 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying lvm2 is present... 2024-05-09T08:18:34.462 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying time synchronization is in place... 2024-05-09T08:18:34.480 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T08:18:34.480 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T08:18:34.496 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T08:18:34.497 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:18:34.517 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2024-05-09T08:18:34.534 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2024-05-09T08:18:34.534 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2024-05-09T08:18:34.534 INFO:teuthology.orchestra.run.smithi133.stdout:Repeating the final host check... 2024-05-09T08:18:35.047 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T08:18:35.047 INFO:teuthology.orchestra.run.smithi133.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T08:18:35.048 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl is present 2024-05-09T08:18:35.048 INFO:teuthology.orchestra.run.smithi133.stdout:lvcreate is present 2024-05-09T08:18:35.067 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T08:18:35.067 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T08:18:35.084 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T08:18:35.084 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:18:35.102 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Host looks OK 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Cluster fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Acquiring lock 140050188877776 on /run/cephadm/99f031e6-0ddc-11ef-bc98-c7b262605968.lock 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Lock 140050188877776 acquired on /run/cephadm/99f031e6-0ddc-11ef-bc98-c7b262605968.lock 2024-05-09T08:18:35.122 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 3300 ... 2024-05-09T08:18:35.123 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 6789 ... 2024-05-09T08:18:35.123 INFO:teuthology.orchestra.run.smithi133.stdout:Base mon IP(s) is [172.21.15.133:3300, 172.21.15.133:6789], mon addrv is [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2024-05-09T08:18:35.131 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.133 metric 100 2024-05-09T08:18:35.131 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.133 metric 100 2024-05-09T08:18:35.140 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T08:18:35.140 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-09T08:18:35.140 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-05-09T08:18:35.149 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T08:18:35.149 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T08:18:35.149 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T08:18:35.149 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-09T08:18:35.150 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-05-09T08:18:35.150 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T08:18:35.151 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2024-05-09T08:18:35.151 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2024-05-09T08:18:35.151 INFO:teuthology.orchestra.run.smithi133.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T08:18:35.154 INFO:teuthology.orchestra.run.smithi133.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T08:18:35.156 INFO:teuthology.orchestra.run.smithi133.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:18:37.812 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T08:18:37.812 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc... 2024-05-09T08:18:37.813 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T08:18:37.813 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T08:18:37.813 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:9852312f87792557815c3e860a5c4f8cbf2b59bbbde9b5f86ab07b438f905b2d 2024-05-09T08:18:37.813 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying config sha256:d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 2024-05-09T08:18:37.813 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T08:18:39.028 INFO:teuthology.orchestra.run.smithi133.stdout:ceph: stdout ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T08:18:39.028 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph version: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev) 2024-05-09T08:18:39.028 INFO:teuthology.orchestra.run.smithi133.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T08:18:40.144 INFO:teuthology.orchestra.run.smithi133.stdout:stat: stdout 167 167 2024-05-09T08:18:40.144 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial keys... 2024-05-09T08:18:41.236 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQDghjxmQOB4IxAA0mvP/EQHhc5RonJsU6tL4g== 2024-05-09T08:18:42.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQDhhjxmiWLBJxAA4e7b1HCP6OxslGx9tfR3zg== 2024-05-09T08:18:44.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQDjhjxmAiiGAhAAsi7wznBVwk2Or7ZM0tbKlg== 2024-05-09T08:18:44.564 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial monmap... 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool for a [v2:172.21.15.133:3300,v1:172.21.15.133:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: set fsid to 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:18:45.581 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mon... 2024-05-09T08:18:46.786 INFO:teuthology.orchestra.run.smithi133.stdout:create mon.a on 2024-05-09T08:18:47.152 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T08:18:47.395 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T08:18:47.681 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target. 2024-05-09T08:18:47.681 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target. 2024-05-09T08:18:48.068 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a 2024-05-09T08:18:48.068 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-09T08:18:48.366 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target.wants/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@.service. 2024-05-09T08:18:49.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 podman[29221]: 2024-05-09 08:18:49.093771681 +0000 UTC m=+0.463898298 container init 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux ) 2024-05-09T08:18:49.227 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:18:49.228 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-05-09T08:18:49.243 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:18:49.244 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:18:49.244 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-05-09T08:18:49.244 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T08:18:49.245 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon to start... 2024-05-09T08:18:49.245 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon... 2024-05-09T08:18:49.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 podman[29221]: 2024-05-09 08:18:49.104989477 +0000 UTC m=+0.475116088 container start 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc) 2024-05-09T08:18:49.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 ceph-mon[29246]: mkfs 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:49.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 ceph-mon[29246]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:18:49.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 bash[29221]: 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb 2024-05-09T08:18:49.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:49 smithi133 systemd[1]: Started Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: monmap epoch 1 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: last_changed 2024-05-09T08:18:44.970809+0000 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:18:50.468 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: min_mon_release 19 (squid) 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: election_strategy: 1 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: fsmap 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: mgrmap e1: no daemons active 2024-05-09T08:18:50.469 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:50 smithi133 ceph-mon[29246]: from='client.? 172.21.15.133:0/1660655453' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout id: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout services: 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.74892s) 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout data: 2024-05-09T08:18:50.625 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.830+0000 7f76bea00700 1 Processor -- start 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.830+0000 7f76bea00700 1 -- start start 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.831+0000 7f76bea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.831+0000 7f76bea00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8108710 con 0x7f76b8107d20 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.832+0000 7f76b7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.832+0000 7f76b7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55624/0 (socket says 172.21.15.133:55624) 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.832+0000 7f76b7e00700 1 -- 172.21.15.133:0/1821530714 learned_addr learned my addr 172.21.15.133:0/1821530714 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.832+0000 7f76b7e00700 1 -- 172.21.15.133:0/1821530714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b8108f20 con 0x7f76b8107d20 2024-05-09T08:18:50.626 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.832+0000 7f76b7e00700 1 --2- 172.21.15.133:0/1821530714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098a0 tx=0x7f76a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359f8c982514bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76b6a00700 1 -- 172.21.15.133:0/1821530714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a8004160 con 0x7f76b8107d20 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76b6a00700 1 -- 172.21.15.133:0/1821530714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8004300 con 0x7f76b8107d20 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76b6a00700 1 -- 172.21.15.133:0/1821530714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a80036a0 con 0x7f76b8107d20 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76bea00700 1 -- 172.21.15.133:0/1821530714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 msgr2=0x7f76b8108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76bea00700 1 --2- 172.21.15.133:0/1821530714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f76a80098a0 tx=0x7f76a8009bb0 comp rx=0 tx=0).stop 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76bea00700 1 -- 172.21.15.133:0/1821530714 shutdown_connections 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76bea00700 1 --2- 172.21.15.133:0/1821530714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b8108140 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.833+0000 7f76bea00700 1 -- 172.21.15.133:0/1821530714 >> 172.21.15.133:0/1821530714 conn(0x7f76b81032e0 msgr2=0x7f76b8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.834+0000 7f76bea00700 1 -- 172.21.15.133:0/1821530714 shutdown_connections 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.834+0000 7f76bea00700 1 -- 172.21.15.133:0/1821530714 wait complete. 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.834+0000 7f76bea00700 1 Processor -- start 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76bea00700 1 -- start start 2024-05-09T08:18:50.627 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76bea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76bea00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8109450 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76b7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76b7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55638/0 (socket says 172.21.15.133:55638) 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.835+0000 7f76b7e00700 1 -- 172.21.15.133:0/1660655453 learned_addr learned my addr 172.21.15.133:0/1660655453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76b7e00700 1 -- 172.21.15.133:0/1660655453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b819db20 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76b7e00700 1 --2- 172.21.15.133:0/1660655453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c00 tx=0x7f76a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a8003db0 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8011630 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76a8011950 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b819ddb0 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.836+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a0aa0 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.837+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f76a801a460 con 0x7f76b8107d20 2024-05-09T08:18:50.628 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.837+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f76a801b780 con 0x7f76b8107d20 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.837+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b8063980 con 0x7f76b8107d20 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.841+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f76a801b980 con 0x7f76b8107d20 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.921+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f76b81a0c70 con 0x7f76b8107d20 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.921+0000 7f76b4c00700 1 -- 172.21.15.133:0/1660655453 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f76a801bb60 con 0x7f76b8107d20 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.923+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 msgr2=0x7f76b819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.923+0000 7f76bea00700 1 --2- 172.21.15.133:0/1660655453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f76a8000c00 tx=0x7f76a8004750 comp rx=0 tx=0).stop 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.923+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 shutdown_connections 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.923+0000 7f76bea00700 1 --2- 172.21.15.133:0/1660655453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b8107d20 0x7f76b819d5e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.923+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 >> 172.21.15.133:0/1660655453 conn(0x7f76b81032e0 msgr2=0x7f76b807d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.924+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 shutdown_connections 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:49.924+0000 7f76bea00700 1 -- 172.21.15.133:0/1660655453 wait complete. 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:mon is available 2024-05-09T08:18:50.629 INFO:teuthology.orchestra.run.smithi133.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T08:18:51.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:51 smithi133 ceph-mon[29246]: from='client.? 172.21.15.133:0/152615801' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T08:18:51.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:51 smithi133 ceph-mon[29246]: from='client.? 172.21.15.133:0/152615801' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T08:18:51.992 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.246+0000 7f8a19200700 1 Processor -- start 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.246+0000 7f8a19200700 1 -- start start 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.247+0000 7f8a19200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.247+0000 7f8a19200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14111600 con 0x7f8a1410ecd0 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.247+0000 7f8a11e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.247+0000 7f8a11e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55646/0 (socket says 172.21.15.133:55646) 2024-05-09T08:18:51.993 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.247+0000 7f8a11e00700 1 -- 172.21.15.133:0/1943624079 learned_addr learned my addr 172.21.15.133:0/1943624079 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.248+0000 7f8a11e00700 1 -- 172.21.15.133:0/1943624079 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a14111780 con 0x7f8a1410ecd0 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.248+0000 7f8a11e00700 1 --2- 172.21.15.133:0/1943624079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a040098a0 tx=0x7f8a04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10ca0c9e42926b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.248+0000 7f8a10a00700 1 -- 172.21.15.133:0/1943624079 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04004160 con 0x7f8a1410ecd0 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.248+0000 7f8a10a00700 1 -- 172.21.15.133:0/1943624079 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04004300 con 0x7f8a1410ecd0 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.248+0000 7f8a10a00700 1 -- 172.21.15.133:0/1943624079 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a040036a0 con 0x7f8a1410ecd0 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 -- 172.21.15.133:0/1943624079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 msgr2=0x7f8a141110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 --2- 172.21.15.133:0/1943624079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a040098a0 tx=0x7f8a04009bb0 comp rx=0 tx=0).stop 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 -- 172.21.15.133:0/1943624079 shutdown_connections 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 --2- 172.21.15.133:0/1943624079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141110c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 -- 172.21.15.133:0/1943624079 >> 172.21.15.133:0/1943624079 conn(0x7f8a1407d2e0 msgr2=0x7f8a1407f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 -- 172.21.15.133:0/1943624079 shutdown_connections 2024-05-09T08:18:51.994 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.249+0000 7f8a19200700 1 -- 172.21.15.133:0/1943624079 wait complete. 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.250+0000 7f8a19200700 1 Processor -- start 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.250+0000 7f8a19200700 1 -- start start 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.250+0000 7f8a19200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.250+0000 7f8a19200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14111b90 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.251+0000 7f8a11e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.251+0000 7f8a11e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55654/0 (socket says 172.21.15.133:55654) 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.251+0000 7f8a11e00700 1 -- 172.21.15.133:0/152615801 learned_addr learned my addr 172.21.15.133:0/152615801 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.251+0000 7f8a11e00700 1 -- 172.21.15.133:0/152615801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a1e30 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.251+0000 7f8a11e00700 1 --2- 172.21.15.133:0/152615801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.252+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04003db0 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.252+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04011630 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.252+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a04011950 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.252+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141a20c0 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.252+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a4db0 con 0x7f8a1410ecd0 2024-05-09T08:18:51.995 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.253+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a04011af0 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.253+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8a0401c450 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.253+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a14063980 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.256+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8a04004260 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.332+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8a141a4f80 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.339+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f8a0401b450 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a0aa00700 1 -- 172.21.15.133:0/152615801 <== mon.0 v2:172.21.15.133:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a0401b630 con 0x7f8a1410ecd0 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 msgr2=0x7f8a141a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 --2- 172.21.15.133:0/152615801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a04004750 comp rx=0 tx=0).stop 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 shutdown_connections 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 --2- 172.21.15.133:0/152615801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1410ecd0 0x7f8a141a18f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 >> 172.21.15.133:0/152615801 conn(0x7f8a1407d2e0 msgr2=0x7f8a1407e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.341+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 shutdown_connections 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:51.342+0000 7f8a19200700 1 -- 172.21.15.133:0/152615801 wait complete. 2024-05-09T08:18:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:Generating new minimal ceph.conf... 2024-05-09T08:18:52.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:52 smithi133 ceph-mon[29246]: from='client.? 172.21.15.133:0/3039383649' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:18:53.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.580+0000 7f8b23e00700 1 Processor -- start 2024-05-09T08:18:53.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.580+0000 7f8b23e00700 1 -- start start 2024-05-09T08:18:53.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.581+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:53.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.581+0000 7f8b23e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24108580 con 0x7f8b24107b90 2024-05-09T08:18:53.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.581+0000 7f8b22a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.581+0000 7f8b22a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55662/0 (socket says 172.21.15.133:55662) 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.581+0000 7f8b22a00700 1 -- 172.21.15.133:0/4119527910 learned_addr learned my addr 172.21.15.133:0/4119527910 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.582+0000 7f8b22a00700 1 -- 172.21.15.133:0/4119527910 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b24108d90 con 0x7f8b24107b90 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.582+0000 7f8b22a00700 1 --2- 172.21.15.133:0/4119527910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b140098a0 tx=0x7f8b14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24082f889adc95d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.582+0000 7f8b21600700 1 -- 172.21.15.133:0/4119527910 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b14004160 con 0x7f8b24107b90 2024-05-09T08:18:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b21600700 1 -- 172.21.15.133:0/4119527910 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b14004300 con 0x7f8b24107b90 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b21600700 1 -- 172.21.15.133:0/4119527910 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b14011b50 con 0x7f8b24107b90 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 -- 172.21.15.133:0/4119527910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 msgr2=0x7f8b24107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 --2- 172.21.15.133:0/4119527910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b140098a0 tx=0x7f8b14009bb0 comp rx=0 tx=0).stop 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 -- 172.21.15.133:0/4119527910 shutdown_connections 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 --2- 172.21.15.133:0/4119527910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b24107fb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 -- 172.21.15.133:0/4119527910 >> 172.21.15.133:0/4119527910 conn(0x7f8b24103170 msgr2=0x7f8b24105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.583+0000 7f8b23e00700 1 -- 172.21.15.133:0/4119527910 shutdown_connections 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.584+0000 7f8b23e00700 1 -- 172.21.15.133:0/4119527910 wait complete. 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b23e00700 1 Processor -- start 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b23e00700 1 -- start start 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b23e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b23e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2410c140 con 0x7f8b24107b90 2024-05-09T08:18:53.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b22a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b22a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55670/0 (socket says 172.21.15.133:55670) 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.585+0000 7f8b22a00700 1 -- 172.21.15.133:0/3039383649 learned_addr learned my addr 172.21.15.133:0/3039383649 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b22a00700 1 -- 172.21.15.133:0/3039383649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2419c000 con 0x7f8b24107b90 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b22a00700 1 --2- 172.21.15.133:0/3039383649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b14011850 tx=0x7f8b14011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b14003af0 con 0x7f8b24107b90 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b14020070 con 0x7f8b24107b90 2024-05-09T08:18:53.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b140195a0 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2419c290 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.586+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2419ef80 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.587+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b14019be0 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.587+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b14011e00 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.588+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b24069440 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.590+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8b14003c90 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.667+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8b2419f660 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.667+0000 7f8b1b400700 1 -- 172.21.15.133:0/3039383649 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f8b1401a430 con 0x7f8b24107b90 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 msgr2=0x7f8b2419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:53.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 --2- 172.21.15.133:0/3039383649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b14011850 tx=0x7f8b14011c10 comp rx=0 tx=0).stop 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 shutdown_connections 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 --2- 172.21.15.133:0/3039383649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b24107b90 0x7f8b2419bac0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 >> 172.21.15.133:0/3039383649 conn(0x7f8b24103170 msgr2=0x7f8b24103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 shutdown_connections 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:52.669+0000 7f8b23e00700 1 -- 172.21.15.133:0/3039383649 wait complete. 2024-05-09T08:18:53.349 INFO:teuthology.orchestra.run.smithi133.stdout:Restarting the monitor... 2024-05-09T08:18:53.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:53 smithi133 systemd[1]: Stopping Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:18:54.182 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a[29240]: 2024-05-09T08:18:53.821+0000 7fc54e600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:18:54.182 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a[29240]: 2024-05-09T08:18:53.821+0000 7fc54e600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T08:18:54.182 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:53 smithi133 podman[29489]: 2024-05-09 08:18:53.904143107 +0000 UTC m=+0.207609374 container died 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:18:54.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:54 smithi133 podman[29489]: 2024-05-09 08:18:54.464657377 +0000 UTC m=+0.768123670 container cleanup 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:18:54.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:54 smithi133 bash[29489]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a 2024-05-09T08:18:54.974 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:54 smithi133 podman[29503]: 2024-05-09 08:18:54.715882664 +0000 UTC m=+0.799070644 container remove 92b7f83dc46131663281311e626c33561b450f0c2a1438d89425b41f0fc47fcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T08:18:55.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 systemd[1]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T08:18:55.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 systemd[1]: Stopped Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:18:55.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 systemd[1]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service: Consumed 1.856s CPU time. 2024-05-09T08:18:55.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 systemd[1]: Starting Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:18:55.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 podman[29591]: 2024-05-09 08:18:55.492403671 +0000 UTC m=+0.034886259 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:18:55.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 podman[29591]: 2024-05-09 08:18:55.608101422 +0000 UTC m=+0.150583997 container create 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, ceph=True) 2024-05-09T08:18:55.967 INFO:teuthology.orchestra.run.smithi133.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T08:18:56.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 podman[29591]: 2024-05-09 08:18:55.845006481 +0000 UTC m=+0.387489103 container init 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T08:18:56.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 podman[29591]: 2024-05-09 08:18:55.854497123 +0000 UTC m=+0.396979720 container start 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T08:18:56.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T08:18:56.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T08:18:56.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: pidfile_write: ignore empty --pid-file 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: load: jerasure load: lrc 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: RocksDB version: 7.9.2 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Git sha 0 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: DB SUMMARY 2024-05-09T08:18:56.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: DB Session ID: Q7L0FIQG6VHNRRRH4RBT 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: CURRENT file: CURRENT 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104920 ; 2024-05-09T08:18:56.124 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.error_if_exists: 0 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.create_if_missing: 0 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.paranoid_checks: 1 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T08:18:56.125 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.env: 0x55eb5f850c60 2024-05-09T08:18:56.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T08:18:56.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.info_log: 0x55eb8ea71b00 2024-05-09T08:18:56.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T08:18:56.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.statistics: (nil) 2024-05-09T08:18:56.129 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.use_fsync: 0 2024-05-09T08:18:56.129 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_log_file_size: 0 2024-05-09T08:18:56.129 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T08:18:56.129 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_fallocate: 1 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T08:18:56.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.use_direct_reads: 0 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.db_log_dir: 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.wal_dir: 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T08:18:56.131 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T08:18:56.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.write_buffer_manager: 0x55eb8ea441e0 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T08:18:56.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.unordered_write: 0 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T08:18:56.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.row_cache: None 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.wal_filter: None 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T08:18:56.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.two_write_queues: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.wal_compression: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.atomic_flush: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T08:18:56.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.log_readahead_size: 0 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T08:18:56.137 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_background_jobs: 2 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_background_compactions: -1 2024-05-09T08:18:56.138 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_subcompactions: 1 2024-05-09T08:18:56.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T08:18:56.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T08:18:56.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T08:18:56.139 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_open_files: -1 2024-05-09T08:18:56.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_background_flushes: -1 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Compression algorithms supported: 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kZSTD supported: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kXpressCompression supported: 0 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T08:18:56.141 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kZlibCompression supported: 1 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kSnappyCompression supported: 1 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kLZ4Compression supported: 1 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: kBZip2Compression supported: 0 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T08:18:56.142 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.merge_operator: 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_filter: None 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eb8ea71c20) 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks: 1 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_top_level_index_and_filter: 1 2024-05-09T08:18:56.143 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_type: 0 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_index_type: 0 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_shortening: 1 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: checksum: 4 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: no_block_cache: 0 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache: 0x55eb8ea75090 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_name: BinnedLRUCache 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_options: 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: capacity : 536870912 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_shard_bits : 4 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: strict_capacity_limit : 0 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: high_pri_pool_ratio: 0.000 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_compressed: (nil) 2024-05-09T08:18:56.144 INFO:journalctl@ceph.mon.a.smithi133.stdout: persistent_cache: (nil) 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size: 4096 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size_deviation: 10 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_restart_interval: 16 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_block_restart_interval: 1 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: metadata_block_size: 4096 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: partition_filters: 0 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: use_delta_encoding: 1 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: filter_policy: bloomfilter 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: whole_key_filtering: 1 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: verify_compression: 0 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: read_amp_bytes_per_bit: 0 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: format_version: 5 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_index_compression: 1 2024-05-09T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_align: 0 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout: max_auto_readahead_size: 262144 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout: prepopulate_block_cache: 0 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout: initial_auto_readahead_size: 8192 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression: NoCompression 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.num_levels: 7 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T08:18:56.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T08:18:56.147 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T08:18:56.148 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T08:18:56.149 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.table_properties_collectors: 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.inplace_update_support: 0 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T08:18:56.150 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.bloom_locality: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.max_successive_merges: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.ttl: 2592000 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enable_blob_files: false 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.min_blob_size: 0 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T08:18:56.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5faeadc7-8910-4597-9713-dd8a40903f04 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242735910196, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242735912176, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97943, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14195, "raw_average_key_size": 49, "raw_value_size": 90230, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715242735, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5faeadc7-8910-4597-9713-dd8a40903f04", "db_session_id": "Q7L0FIQG6VHNRRRH4RBT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242735912369, "job": 1, "event": "recovery_finished"} 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T08:18:56.152 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eb8eb66000 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: DB pointer 0x55eb8eb50000 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** DB Stats ** 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:18:56.153 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: L0 2/0 99.52 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: Sum 2/0 99.52 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T08:18:56.154 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative compaction: 0.00 GB write, 14.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval compaction: 0.00 GB write, 14.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache BinnedLRUCache@0x55eb8ea75090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T08:18:56.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: starting mon.a rank 0 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???) e1 preinit fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).mds e1 new map 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).mds e1 print_map 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: e1 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: legacy client fscid: -1 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout: No filesystems configured 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T08:18:56.156 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 bash[29591]: 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 systemd[1]: Started Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: monmap epoch 1 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: last_changed 2024-05-09T08:18:44.970809+0000 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: min_mon_release 19 (squid) 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: election_strategy: 1 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: fsmap 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T08:18:56.157 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:55 smithi133 ceph-mon[29610]: mgrmap e1: no daemons active 2024-05-09T08:18:57.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.659+0000 7f0cc3a00700 1 Processor -- start 2024-05-09T08:18:57.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.659+0000 7f0cc3a00700 1 -- start start 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.660+0000 7f0cc3a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.660+0000 7f0cc3a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc10a880 con 0x7f0cbc109f20 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.661+0000 7f0cc1000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.661+0000 7f0cc1000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35434/0 (socket says 172.21.15.133:35434) 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.661+0000 7f0cc1000700 1 -- 172.21.15.133:0/3068208077 learned_addr learned my addr 172.21.15.133:0/3068208077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.661+0000 7f0cc1000700 1 -- 172.21.15.133:0/3068208077 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc10b080 con 0x7f0cbc109f20 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.661+0000 7f0cc1000700 1 --2- 172.21.15.133:0/3068208077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098a0 tx=0x7f0cb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e006e29cc7f74eab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.662+0000 7f0cbb400700 1 -- 172.21.15.133:0/3068208077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0004160 con 0x7f0cbc109f20 2024-05-09T08:18:57.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.662+0000 7f0cbb400700 1 -- 172.21.15.133:0/3068208077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cb0004300 con 0x7f0cbc109f20 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.662+0000 7f0cbb400700 1 -- 172.21.15.133:0/3068208077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0011b50 con 0x7f0cbc109f20 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.662+0000 7f0cc3a00700 1 -- 172.21.15.133:0/3068208077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 msgr2=0x7f0cbc10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 --2- 172.21.15.133:0/3068208077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098a0 tx=0x7f0cb0009bb0 comp rx=0 tx=0).stop 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 -- 172.21.15.133:0/3068208077 shutdown_connections 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 --2- 172.21.15.133:0/3068208077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc10a340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 -- 172.21.15.133:0/3068208077 >> 172.21.15.133:0/3068208077 conn(0x7f0cbc07d2d0 msgr2=0x7f0cbc07d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 -- 172.21.15.133:0/3068208077 shutdown_connections 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.663+0000 7f0cc3a00700 1 -- 172.21.15.133:0/3068208077 wait complete. 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.664+0000 7f0cc3a00700 1 Processor -- start 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.664+0000 7f0cc3a00700 1 -- start start 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.665+0000 7f0cc3a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.665+0000 7f0cc3a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc10e4c0 con 0x7f0cbc109f20 2024-05-09T08:18:57.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.665+0000 7f0cc1000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.665+0000 7f0cc1000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35436/0 (socket says 172.21.15.133:35436) 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.665+0000 7f0cc1000700 1 -- 172.21.15.133:0/2157176982 learned_addr learned my addr 172.21.15.133:0/2157176982 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.666+0000 7f0cc1000700 1 -- 172.21.15.133:0/2157176982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc19e2d0 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.666+0000 7f0cc1000700 1 --2- 172.21.15.133:0/2157176982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0011b60 tx=0x7f0cb0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.666+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0003b20 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cb0003cc0 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0024840 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc19e560 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1a1210 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0cb0022020 con 0x7f0cbc109f20 2024-05-09T08:18:57.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.667+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0cb0011e00 con 0x7f0cbc109f20 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.668+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc063980 con 0x7f0cbc109f20 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.671+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0cb001a430 con 0x7f0cbc109f20 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.747+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f0cbc1a1870 con 0x7f0cbc109f20 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.750+0000 7f0cb9600700 1 -- 172.21.15.133:0/2157176982 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f0cb0032030 con 0x7f0cbc109f20 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.751+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 msgr2=0x7f0cbc19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.751+0000 7f0cc3a00700 1 --2- 172.21.15.133:0/2157176982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0011b60 tx=0x7f0cb0011c40 comp rx=0 tx=0).stop 2024-05-09T08:18:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.751+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 shutdown_connections 2024-05-09T08:18:57.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.751+0000 7f0cc3a00700 1 --2- 172.21.15.133:0/2157176982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cbc109f20 0x7f0cbc19dd90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:18:57.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.751+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 >> 172.21.15.133:0/2157176982 conn(0x7f0cbc07d2d0 msgr2=0x7f0cbc107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:18:57.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.752+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 shutdown_connections 2024-05-09T08:18:57.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:56.752+0000 7f0cc3a00700 1 -- 172.21.15.133:0/2157176982 wait complete. 2024-05-09T08:18:57.348 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T08:18:57.385 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:18:57.385 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mgr... 2024-05-09T08:18:57.387 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T08:18:57.388 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T08:18:57.828 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x 2024-05-09T08:18:57.829 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-09T08:18:58.080 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:57 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2157176982' entity='client.admin' 2024-05-09T08:18:58.101 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968.target.wants/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@.service. 2024-05-09T08:18:58.430 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 systemd[1]: Starting Ceph mgr.x for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:18:58.768 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 podman[29822]: 2024-05-09 08:18:58.401642346 +0000 UTC m=+0.034896331 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:18:58.768 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 podman[29822]: 2024-05-09 08:18:58.519491088 +0000 UTC m=+0.152745110 container create a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T08:18:58.768 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 podman[29822]: 2024-05-09 08:18:58.764469238 +0000 UTC m=+0.397723270 container init a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T08:18:58.886 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:18:58.886 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-05-09T08:18:58.904 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:18:58.905 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:18:58.905 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-05-09T08:18:58.905 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T08:18:58.933 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:18:58.933 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr to start... 2024-05-09T08:18:58.954 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr... 2024-05-09T08:18:59.040 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 podman[29822]: 2024-05-09 08:18:58.774987985 +0000 UTC m=+0.408241995 container start a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T08:18:59.040 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 bash[29822]: a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 2024-05-09T08:18:59.040 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:58 smithi133 systemd[1]: Started Ceph mgr.x for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:18:59.294 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:18:59.037+0000 7fd35326f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:18:59.564 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:18:59.298+0000 7fd35326f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:18:59.564 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:18:59.436+0000 7fd35326f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:18:59.861 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:18:59 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2186008754' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:18:59.862 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:18:59.858+0000 7fd35326f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:19:00.180 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:18:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:18:59.974+0000 7fd35326f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:19:00.300 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:00.300 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:19:00.301 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:19:00.302 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:19:00.303 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:19:00.312 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:19:00.312 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:19:00.312 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:19:00.312 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:19:00.313 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:19:00.314 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:18:49.175297+0000", 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.560+0000 7fe2a6000700 1 Processor -- start 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.560+0000 7fe2a6000700 1 -- start start 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.561+0000 7fe2a6000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.561+0000 7fe2a6000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a007bdf0 con 0x7fe2a01086d0 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.562+0000 7fe2a4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.562+0000 7fe2a4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35478/0 (socket says 172.21.15.133:35478) 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.562+0000 7fe2a4c00700 1 -- 172.21.15.133:0/801245819 learned_addr learned my addr 172.21.15.133:0/801245819 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:00.315 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.562+0000 7fe2a4c00700 1 -- 172.21.15.133:0/801245819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a010b000 con 0x7fe2a01086d0 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.562+0000 7fe2a4c00700 1 --2- 172.21.15.133:0/801245819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2940098a0 tx=0x7fe294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a2b15f15ec511 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.563+0000 7fe29f400700 1 -- 172.21.15.133:0/801245819 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe294004160 con 0x7fe2a01086d0 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.563+0000 7fe29f400700 1 -- 172.21.15.133:0/801245819 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe294004300 con 0x7fe2a01086d0 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.563+0000 7fe29f400700 1 -- 172.21.15.133:0/801245819 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe294011b50 con 0x7fe2a01086d0 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.563+0000 7fe2a6000700 1 -- 172.21.15.133:0/801245819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 msgr2=0x7fe2a010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.563+0000 7fe2a6000700 1 --2- 172.21.15.133:0/801245819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2940098a0 tx=0x7fe294009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.564+0000 7fe2a6000700 1 -- 172.21.15.133:0/801245819 shutdown_connections 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.564+0000 7fe2a6000700 1 --2- 172.21.15.133:0/801245819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a010aac0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.564+0000 7fe2a6000700 1 -- 172.21.15.133:0/801245819 >> 172.21.15.133:0/801245819 conn(0x7fe2a0102060 msgr2=0x7fe2a0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.564+0000 7fe2a6000700 1 -- 172.21.15.133:0/801245819 shutdown_connections 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.564+0000 7fe2a6000700 1 -- 172.21.15.133:0/801245819 wait complete. 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a6000700 1 Processor -- start 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a6000700 1 -- start start 2024-05-09T08:19:00.316 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a6000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a6000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a010bf70 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35488/0 (socket says 172.21.15.133:35488) 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.565+0000 7fe2a4c00700 1 -- 172.21.15.133:0/2186008754 learned_addr learned my addr 172.21.15.133:0/2186008754 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe2a4c00700 1 -- 172.21.15.133:0/2186008754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a019e1c0 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe2a4c00700 1 --2- 172.21.15.133:0/2186008754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe294011b60 tx=0x7fe294011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe294003970 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe294003b10 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a019e450 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.566+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe294024610 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.567+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01a1140 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.567+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe294024c50 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.567+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe294011e00 con 0x7fe2a01086d0 2024-05-09T08:19:00.317 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.568+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a0069440 con 0x7fe2a01086d0 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.571+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe29401a770 con 0x7fe2a01086d0 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.649+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe2a01a1310 con 0x7fe2a01086d0 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.650+0000 7fe29d600700 1 -- 172.21.15.133:0/2186008754 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe29401a9f0 con 0x7fe2a01086d0 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 msgr2=0x7fe2a019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 --2- 172.21.15.133:0/2186008754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe294011b60 tx=0x7fe294011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 shutdown_connections 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 --2- 172.21.15.133:0/2186008754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2a01086d0 0x7fe2a019dc80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 >> 172.21.15.133:0/2186008754 conn(0x7fe2a0102060 msgr2=0x7fe2a0102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 shutdown_connections 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:18:59.652+0000 7fe2a6000700 1 -- 172.21.15.133:0/2186008754 wait complete. 2024-05-09T08:19:00.318 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (1/15)... 2024-05-09T08:19:01.930 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:01.537+0000 7fd35326f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:19:01.930 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:01.758+0000 7fd35326f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:02.360 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:01.980+0000 7fd35326f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:19:02.361 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:02.358+0000 7fd35326f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:19:02.640 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:02.482+0000 7fd35326f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:19:02.910 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:02.740+0000 7fd35326f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:19:03.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:03 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2391955428' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:19:03.180 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:02.998+0000 7fd35326f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:19:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:19:03.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:19:03.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:19:03.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:19:03.664 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:19:03.665 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:18:49.175297+0000", 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.906+0000 7fc9e0a00700 1 Processor -- start 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.906+0000 7fc9e0a00700 1 -- start start 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.907+0000 7fc9e0a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:03.666 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.907+0000 7fc9e0a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc07c8f0 con 0x7fc9dc07de70 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.908+0000 7fc9d9e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.908+0000 7fc9d9e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35498/0 (socket says 172.21.15.133:35498) 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.908+0000 7fc9d9e00700 1 -- 172.21.15.133:0/3915504086 learned_addr learned my addr 172.21.15.133:0/3915504086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.908+0000 7fc9d9e00700 1 -- 172.21.15.133:0/3915504086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc07ca70 con 0x7fc9dc07de70 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.909+0000 7fc9d9e00700 1 --2- 172.21.15.133:0/3915504086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098a0 tx=0x7fc9cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a5a0db7b348b8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.909+0000 7fc9d8a00700 1 -- 172.21.15.133:0/3915504086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9cc004160 con 0x7fc9dc07de70 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.909+0000 7fc9d8a00700 1 -- 172.21.15.133:0/3915504086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9cc004300 con 0x7fc9dc07de70 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.909+0000 7fc9d8a00700 1 -- 172.21.15.133:0/3915504086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9cc011b50 con 0x7fc9dc07de70 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 -- 172.21.15.133:0/3915504086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 msgr2=0x7fc9dc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 --2- 172.21.15.133:0/3915504086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098a0 tx=0x7fc9cc009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 -- 172.21.15.133:0/3915504086 shutdown_connections 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 --2- 172.21.15.133:0/3915504086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc07c320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 -- 172.21.15.133:0/3915504086 >> 172.21.15.133:0/3915504086 conn(0x7fc9dc1032e0 msgr2=0x7fc9dc105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:03.667 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 -- 172.21.15.133:0/3915504086 shutdown_connections 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.910+0000 7fc9e0a00700 1 -- 172.21.15.133:0/3915504086 wait complete. 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.911+0000 7fc9e0a00700 1 Processor -- start 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9e0a00700 1 -- start start 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9e0a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9e0a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc10e2d0 con 0x7fc9dc07de70 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9d9e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9d9e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35500/0 (socket says 172.21.15.133:35500) 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.912+0000 7fc9d9e00700 1 -- 172.21.15.133:0/2391955428 learned_addr learned my addr 172.21.15.133:0/2391955428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.913+0000 7fc9d9e00700 1 -- 172.21.15.133:0/2391955428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc19e330 con 0x7fc9dc07de70 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.913+0000 7fc9d9e00700 1 --2- 172.21.15.133:0/2391955428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc011b60 tx=0x7fc9cc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.913+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9cc003b20 con 0x7fc9dc07de70 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.913+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9cc003cc0 con 0x7fc9dc07de70 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.913+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc19e5c0 con 0x7fc9dc07de70 2024-05-09T08:19:03.668 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.914+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9cc024840 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.914+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc1a12b0 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.914+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc9cc022020 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.914+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc9cc011e00 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.915+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9dc063980 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.919+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc9cc043dd0 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.998+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc9dc1a1480 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:02.999+0000 7fc9d2a00700 1 -- 172.21.15.133:0/2391955428 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc9cc032030 con 0x7fc9dc07de70 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 msgr2=0x7fc9dc19ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 --2- 172.21.15.133:0/2391955428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc011b60 tx=0x7fc9cc011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 shutdown_connections 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 --2- 172.21.15.133:0/2391955428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9dc07de70 0x7fc9dc19ddf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 >> 172.21.15.133:0/2391955428 conn(0x7fc9dc1032e0 msgr2=0x7fc9dc103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 shutdown_connections 2024-05-09T08:19:03.669 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:03.000+0000 7fc9e0a00700 1 -- 172.21.15.133:0/2391955428 wait complete. 2024-05-09T08:19:03.670 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (2/15)... 2024-05-09T08:19:03.930 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:03.814+0000 7fd35326f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:19:04.679 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:04.360+0000 7fd35326f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:19:04.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:04.486+0000 7fd35326f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:19:05.180 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:04.743+0000 7fd35326f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:19:05.679 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:05.379+0000 7fd35326f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:19:05.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:05.500+0000 7fd35326f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:19:06.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:05.819+0000 7fd35326f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:19:06.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:06.005+0000 7fd35326f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:19:06.354 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:06.130+0000 7fd35326f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:19:06.354 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:06.287+0000 7fd35326f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:06.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:06 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/941762166' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:19:06.930 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:06.672+0000 7fd35326f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:19:06.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:19:06.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:19:06.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:19:06.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:06.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:19:06.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:19:06.996 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:19:06.997 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:18:49.175297+0000", 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:06.998 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.245+0000 7fcc1b200700 1 Processor -- start 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.245+0000 7fcc1b200700 1 -- start start 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.245+0000 7fcc1b200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.245+0000 7fcc1b200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c108580 con 0x7fcc1c107b90 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.246+0000 7fcc19e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.246+0000 7fcc19e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39366/0 (socket says 172.21.15.133:39366) 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.246+0000 7fcc19e00700 1 -- 172.21.15.133:0/2592691194 learned_addr learned my addr 172.21.15.133:0/2592691194 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.246+0000 7fcc19e00700 1 -- 172.21.15.133:0/2592691194 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c108d90 con 0x7fcc1c107b90 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.247+0000 7fcc19e00700 1 --2- 172.21.15.133:0/2592691194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e80 tx=0x7fcc0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26190247cfa22cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.247+0000 7fcc18a00700 1 -- 172.21.15.133:0/2592691194 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c017070 con 0x7fcc1c107b90 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.247+0000 7fcc18a00700 1 -- 172.21.15.133:0/2592691194 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc0c00b8c0 con 0x7fcc1c107b90 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.247+0000 7fcc18a00700 1 -- 172.21.15.133:0/2592691194 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c0045c0 con 0x7fcc1c107b90 2024-05-09T08:19:06.999 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 -- 172.21.15.133:0/2592691194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 msgr2=0x7fcc1c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 --2- 172.21.15.133:0/2592691194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e80 tx=0x7fcc0c00b330 comp rx=0 tx=0).stop 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 -- 172.21.15.133:0/2592691194 shutdown_connections 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 --2- 172.21.15.133:0/2592691194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c107fb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 -- 172.21.15.133:0/2592691194 >> 172.21.15.133:0/2592691194 conn(0x7fcc1c103170 msgr2=0x7fcc1c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 -- 172.21.15.133:0/2592691194 shutdown_connections 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.248+0000 7fcc1b200700 1 -- 172.21.15.133:0/2592691194 wait complete. 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.249+0000 7fcc1b200700 1 Processor -- start 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.249+0000 7fcc1b200700 1 -- start start 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.249+0000 7fcc1b200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.249+0000 7fcc1b200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10c140 con 0x7fcc1c107b90 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.250+0000 7fcc19e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.250+0000 7fcc19e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39376/0 (socket says 172.21.15.133:39376) 2024-05-09T08:19:07.000 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.250+0000 7fcc19e00700 1 -- 172.21.15.133:0/941762166 learned_addr learned my addr 172.21.15.133:0/941762166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.250+0000 7fcc19e00700 1 -- 172.21.15.133:0/941762166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c19bfb0 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.250+0000 7fcc19e00700 1 --2- 172.21.15.133:0/941762166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e50 tx=0x7fcc0c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c020020 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc0c003b50 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c19c240 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c017040 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c19ef30 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc0c023020 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.251+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc0c00f050 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.252+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c069440 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.255+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcc0c02a020 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.333+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc1c19f100 con 0x7fcc1c107b90 2024-05-09T08:19:07.001 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.334+0000 7fcc12a00700 1 -- 172.21.15.133:0/941762166 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fcc0c003d00 con 0x7fcc1c107b90 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 msgr2=0x7fcc1c19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 --2- 172.21.15.133:0/941762166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009e50 tx=0x7fcc0c00bd30 comp rx=0 tx=0).stop 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 shutdown_connections 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 --2- 172.21.15.133:0/941762166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc1c107b90 0x7fcc1c19ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 >> 172.21.15.133:0/941762166 conn(0x7fcc1c103170 msgr2=0x7fcc1c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.335+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 shutdown_connections 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:06.336+0000 7fcc1b200700 1 -- 172.21.15.133:0/941762166 wait complete. 2024-05-09T08:19:07.002 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (3/15)... 2024-05-09T08:19:07.705 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:07.426+0000 7fd35326f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:19:08.131 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:07.702+0000 7fd35326f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: Activating manager daemon x 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: mgrmap e2: x(active, starting, since 0.0122762s) 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: Manager daemon x is now available 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:19:08.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:19:08.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' 2024-05-09T08:19:08.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:07 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' 2024-05-09T08:19:09.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:08 smithi133 ceph-mon[29610]: from='mgr.14100 172.21.15.133:0/2202860775' entity='mgr.x' 2024-05-09T08:19:09.154 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:08 smithi133 ceph-mon[29610]: mgrmap e3: x(active, since 1.02022s) 2024-05-09T08:19:10.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:10 smithi133 ceph-mon[29610]: mgrmap e4: x(active, since 2s) 2024-05-09T08:19:10.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:10 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/444638331' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T08:19:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:19:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T08:19:10.972 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:10.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T08:19:10.974 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T08:19:10.983 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T08:19:10.984 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T08:18:49.175297+0000", 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.621+0000 7ff808600700 1 Processor -- start 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.621+0000 7ff808600700 1 -- start start 2024-05-09T08:19:10.985 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.621+0000 7ff808600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.621+0000 7ff808600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff800111600 con 0x7ff80010ecd0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.622+0000 7ff805c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.622+0000 7ff805c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39460/0 (socket says 172.21.15.133:39460) 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.622+0000 7ff805c00700 1 -- 172.21.15.133:0/2242601452 learned_addr learned my addr 172.21.15.133:0/2242601452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.622+0000 7ff805c00700 1 -- 172.21.15.133:0/2242601452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff800111780 con 0x7ff80010ecd0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.623+0000 7ff805c00700 1 --2- 172.21.15.133:0/2242601452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40098a0 tx=0x7ff7f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b294040f5fa0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.623+0000 7ff7ffe00700 1 -- 172.21.15.133:0/2242601452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f4004160 con 0x7ff80010ecd0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.623+0000 7ff7ffe00700 1 -- 172.21.15.133:0/2242601452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7f4004300 con 0x7ff80010ecd0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.623+0000 7ff7ffe00700 1 -- 172.21.15.133:0/2242601452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f4011b50 con 0x7ff80010ecd0 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 -- 172.21.15.133:0/2242601452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 msgr2=0x7ff8001110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 --2- 172.21.15.133:0/2242601452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40098a0 tx=0x7ff7f4009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 -- 172.21.15.133:0/2242601452 shutdown_connections 2024-05-09T08:19:10.986 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 --2- 172.21.15.133:0/2242601452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001110c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 -- 172.21.15.133:0/2242601452 >> 172.21.15.133:0/2242601452 conn(0x7ff80007d2d0 msgr2=0x7ff80007f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 -- 172.21.15.133:0/2242601452 shutdown_connections 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.624+0000 7ff808600700 1 -- 172.21.15.133:0/2242601452 wait complete. 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.625+0000 7ff808600700 1 Processor -- start 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.625+0000 7ff808600700 1 -- start start 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.625+0000 7ff808600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.625+0000 7ff808600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8001126f0 con 0x7ff80010ecd0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.626+0000 7ff805c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.626+0000 7ff805c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39468/0 (socket says 172.21.15.133:39468) 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.626+0000 7ff805c00700 1 -- 172.21.15.133:0/444638331 learned_addr learned my addr 172.21.15.133:0/444638331 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.626+0000 7ff805c00700 1 -- 172.21.15.133:0/444638331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8001a26c0 con 0x7ff80010ecd0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.626+0000 7ff805c00700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4011b60 tx=0x7ff7f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.627+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f4003b20 con 0x7ff80010ecd0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.627+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7f4003cc0 con 0x7ff80010ecd0 2024-05-09T08:19:10.987 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.627+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8001a2950 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.627+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7f4024840 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.627+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8001a5640 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.628+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7ff7f4022020 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.628+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff800063980 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.628+0000 7ff7fe000700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 0x7ff7e803b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.628+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7f4011e00 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.629+0000 7ff805200700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 0x7ff7e803b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.630+0000 7ff805200700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 0x7ff7e803b4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec006fd0 tx=0x7ff7ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.635+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff7f40172b0 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.771+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff7f402d770 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.944+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff8001a5810 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.945+0000 7ff7fe000700 1 -- 172.21.15.133:0/444638331 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff8001a5810 con 0x7ff80010ecd0 2024-05-09T08:19:10.988 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 msgr2=0x7ff7e803b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 0x7ff7e803b4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec006fd0 tx=0x7ff7ec006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 msgr2=0x7ff8001a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4011b60 tx=0x7ff7f4011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 shutdown_connections 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7ff7e8038ff0 0x7ff7e803b4b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 --2- 172.21.15.133:0/444638331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff80010ecd0 0x7ff8001a2180 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 >> 172.21.15.133:0/444638331 conn(0x7ff80007d2d0 msgr2=0x7ff80007dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 shutdown_connections 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:09.948+0000 7ff808600700 1 -- 172.21.15.133:0/444638331 wait complete. 2024-05-09T08:19:10.989 INFO:teuthology.orchestra.run.smithi133.stdout:mgr is available 2024-05-09T08:19:12.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:11 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3048646618' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T08:19:12.500 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:12.500 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T08:19:12.501 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:19:12.501 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T08:19:12.501 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T08:19:12.501 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T08:19:12.501 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T08:19:12.510 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T08:19:12.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.604+0000 7f0263c00700 1 Processor -- start 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.604+0000 7f0263c00700 1 -- start start 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.605+0000 7f0263c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.605+0000 7f0263c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f025c108730 con 0x7f025c107d40 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.605+0000 7f0261200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:12.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.605+0000 7f0261200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39474/0 (socket says 172.21.15.133:39474) 2024-05-09T08:19:12.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.605+0000 7f0261200700 1 -- 172.21.15.133:0/2888779831 learned_addr learned my addr 172.21.15.133:0/2888779831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:12.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.606+0000 7f0261200700 1 -- 172.21.15.133:0/2888779831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c108ef0 con 0x7f025c107d40 2024-05-09T08:19:12.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.606+0000 7f0261200700 1 --2- 172.21.15.133:0/2888779831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02500098a0 tx=0x7f0250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33afd41a6a5d977a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:12.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f025b400700 1 -- 172.21.15.133:0/2888779831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250004160 con 0x7f025c107d40 2024-05-09T08:19:12.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f025b400700 1 -- 172.21.15.133:0/2888779831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0250004300 con 0x7f025c107d40 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f025b400700 1 -- 172.21.15.133:0/2888779831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250011b50 con 0x7f025c107d40 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f0263c00700 1 -- 172.21.15.133:0/2888779831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 msgr2=0x7f025c108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f0263c00700 1 --2- 172.21.15.133:0/2888779831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02500098a0 tx=0x7f0250009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f0263c00700 1 -- 172.21.15.133:0/2888779831 shutdown_connections 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f0263c00700 1 --2- 172.21.15.133:0/2888779831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c108160 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:12.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.607+0000 7f0263c00700 1 -- 172.21.15.133:0/2888779831 >> 172.21.15.133:0/2888779831 conn(0x7f025c1032e0 msgr2=0x7f025c105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.608+0000 7f0263c00700 1 -- 172.21.15.133:0/2888779831 shutdown_connections 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.608+0000 7f0263c00700 1 -- 172.21.15.133:0/2888779831 wait complete. 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0263c00700 1 Processor -- start 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0263c00700 1 -- start start 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0263c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:12.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0263c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f025c10c2a0 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0261200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0261200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39488/0 (socket says 172.21.15.133:39488) 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.609+0000 7f0261200700 1 -- 172.21.15.133:0/3048646618 learned_addr learned my addr 172.21.15.133:0/3048646618 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0261200700 1 -- 172.21.15.133:0/3048646618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c07e0b0 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0261200700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f0250011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250003b20 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0250003cc0 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0250024910 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c07c2a0 con 0x7f025c107d40 2024-05-09T08:19:12.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.610+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c07c7e0 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.611+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0250022020 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.612+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c063980 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.612+0000 7f0259600700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 0x7f024403b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.612+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0250011e00 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.612+0000 7f025be00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 0x7f024403b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.613+0000 7f025be00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 0x7f024403b5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.617+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f025001ad80 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.819+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f025c07cfc0 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.821+0000 7f0259600700 1 -- 172.21.15.133:0/3048646618 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f0250017ca0 con 0x7f025c107d40 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 msgr2=0x7f024403b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:12.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 0x7f024403b5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0248006fd0 tx=0x7f0248006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 msgr2=0x7f025c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f0250011c10 comp rx=0 tx=0).stop 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 shutdown_connections 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0244039110 0x7f024403b5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 --2- 172.21.15.133:0/3048646618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f025c107d40 0x7f025c07db70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 >> 172.21.15.133:0/3048646618 conn(0x7f025c1032e0 msgr2=0x7f025c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 shutdown_connections 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:11.824+0000 7f0263c00700 1 -- 172.21.15.133:0/3048646618 wait complete. 2024-05-09T08:19:12.518 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling cephadm module... 2024-05-09T08:19:13.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:13 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3790936506' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T08:19:13.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: ignoring --setuser ceph since I am not root 2024-05-09T08:19:13.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: ignoring --setgroup ceph since I am not root 2024-05-09T08:19:14.114 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:13.732+0000 7fe39111e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:19:14.115 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:13.977+0000 7fe39111e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:19:14.115 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:14.111+0000 7fe39111e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:19:14.145 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.102+0000 7f4685c00700 1 Processor -- start 2024-05-09T08:19:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.102+0000 7f4685c00700 1 -- start start 2024-05-09T08:19:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.103+0000 7f4685c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.103+0000 7f4685c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4680108700 con 0x7f4680107d10 2024-05-09T08:19:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.103+0000 7f467ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:14.146 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.103+0000 7f467ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39490/0 (socket says 172.21.15.133:39490) 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.103+0000 7f467ea00700 1 -- 172.21.15.133:0/1603500074 learned_addr learned my addr 172.21.15.133:0/1603500074 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.104+0000 7f467ea00700 1 -- 172.21.15.133:0/1603500074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4680108f10 con 0x7f4680107d10 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.104+0000 7f467ea00700 1 --2- 172.21.15.133:0/1603500074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46700098a0 tx=0x7f4670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6aed9ba47fe0df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.104+0000 7f467d600700 1 -- 172.21.15.133:0/1603500074 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670004160 con 0x7f4680107d10 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.104+0000 7f467d600700 1 -- 172.21.15.133:0/1603500074 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4670004300 con 0x7f4680107d10 2024-05-09T08:19:14.147 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.104+0000 7f467d600700 1 -- 172.21.15.133:0/1603500074 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670011b50 con 0x7f4680107d10 2024-05-09T08:19:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 -- 172.21.15.133:0/1603500074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 msgr2=0x7f4680108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 --2- 172.21.15.133:0/1603500074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f46700098a0 tx=0x7f4670009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 -- 172.21.15.133:0/1603500074 shutdown_connections 2024-05-09T08:19:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 --2- 172.21.15.133:0/1603500074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f4680108130 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:14.148 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 -- 172.21.15.133:0/1603500074 >> 172.21.15.133:0/1603500074 conn(0x7f46801032d0 msgr2=0x7f46801056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 -- 172.21.15.133:0/1603500074 shutdown_connections 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.105+0000 7f4685c00700 1 -- 172.21.15.133:0/1603500074 wait complete. 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.107+0000 7f4685c00700 1 Processor -- start 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.107+0000 7f4685c00700 1 -- start start 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.107+0000 7f4685c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:14.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.107+0000 7f4685c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f468010c2c0 con 0x7f4680107d10 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.108+0000 7f467ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.108+0000 7f467ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39500/0 (socket says 172.21.15.133:39500) 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.108+0000 7f467ea00700 1 -- 172.21.15.133:0/3790936506 learned_addr learned my addr 172.21.15.133:0/3790936506 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.108+0000 7f467ea00700 1 -- 172.21.15.133:0/3790936506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468019e380 con 0x7f4680107d10 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.108+0000 7f467ea00700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4670011670 tx=0x7f4670011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:14.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.109+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46700039c0 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.109+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4670011e50 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.109+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467001a5c0 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.109+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468019e610 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.109+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46801a1300 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.110+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4670024430 con 0x7f4680107d10 2024-05-09T08:19:14.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.110+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4680063980 con 0x7f4680107d10 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.111+0000 7f4677400700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 0x7f466403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.111+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f467004d3f0 con 0x7f4680107d10 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.111+0000 7f467e000700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 0x7f466403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.112+0000 7f467e000700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 0x7f466403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f466c006fd0 tx=0x7f466c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.116+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f467004ce10 con 0x7f4680107d10 2024-05-09T08:19:14.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.401+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f46801a14d0 con 0x7f4680107d10 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.454+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f46801a14d0 con 0x7f4680107d10 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.457+0000 7f4677400700 1 -- 172.21.15.133:0/3790936506 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f467002d3f0 con 0x7f4680107d10 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 msgr2=0x7f466403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 0x7f466403b580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f466c006fd0 tx=0x7f466c006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 msgr2=0x7f468019de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:14.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4670011670 tx=0x7f4670011b60 comp rx=0 tx=0).stop 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 shutdown_connections 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f46640390c0 0x7f466403b580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 --2- 172.21.15.133:0/3790936506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4680107d10 0x7f468019de40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.460+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 >> 172.21.15.133:0/3790936506 conn(0x7f46801032d0 msgr2=0x7f4680103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.461+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 shutdown_connections 2024-05-09T08:19:14.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:13.461+0000 7f4685c00700 1 -- 172.21.15.133:0/3790936506 wait complete. 2024-05-09T08:19:14.744 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:14 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3790936506' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T08:19:14.744 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:14 smithi133 ceph-mon[29610]: mgrmap e5: x(active, since 5s) 2024-05-09T08:19:14.744 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:14.523+0000 7fe39111e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:19:14.745 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:14.637+0000 7fe39111e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:19:15.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:15 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1907689350' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T08:19:15.698 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:15.698 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T08:19:15.699 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:19:15.699 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T08:19:15.699 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T08:19:15.699 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.741+0000 7fbc87a00700 1 Processor -- start 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.742+0000 7fbc87a00700 1 -- start start 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.742+0000 7fbc87a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.742+0000 7fbc87a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8007c800 con 0x7fbc8007de10 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.742+0000 7fbc85000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.743+0000 7fbc85000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33332/0 (socket says 172.21.15.133:33332) 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.743+0000 7fbc85000700 1 -- 172.21.15.133:0/2626164868 learned_addr learned my addr 172.21.15.133:0/2626164868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.743+0000 7fbc85000700 1 -- 172.21.15.133:0/2626164868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8007c980 con 0x7fbc8007de10 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.743+0000 7fbc85000700 1 --2- 172.21.15.133:0/2626164868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6902f41cde05f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:15.700 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.744+0000 7fbc7f400700 1 -- 172.21.15.133:0/2626164868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74004160 con 0x7fbc8007de10 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.744+0000 7fbc7f400700 1 -- 172.21.15.133:0/2626164868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc74004300 con 0x7fbc8007de10 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.744+0000 7fbc7f400700 1 -- 172.21.15.133:0/2626164868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74011b50 con 0x7fbc8007de10 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.744+0000 7fbc87a00700 1 -- 172.21.15.133:0/2626164868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 msgr2=0x7fbc8007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.744+0000 7fbc87a00700 1 --2- 172.21.15.133:0/2626164868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc740098a0 tx=0x7fbc74009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.745+0000 7fbc87a00700 1 -- 172.21.15.133:0/2626164868 shutdown_connections 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.745+0000 7fbc87a00700 1 --2- 172.21.15.133:0/2626164868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8007c2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.745+0000 7fbc87a00700 1 -- 172.21.15.133:0/2626164868 >> 172.21.15.133:0/2626164868 conn(0x7fbc801032e0 msgr2=0x7fbc80105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.745+0000 7fbc87a00700 1 -- 172.21.15.133:0/2626164868 shutdown_connections 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.745+0000 7fbc87a00700 1 -- 172.21.15.133:0/2626164868 wait complete. 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.746+0000 7fbc87a00700 1 Processor -- start 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.746+0000 7fbc87a00700 1 -- start start 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.746+0000 7fbc87a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.746+0000 7fbc87a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8010e330 con 0x7fbc8007de10 2024-05-09T08:19:15.701 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.747+0000 7fbc85000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.747+0000 7fbc85000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33336/0 (socket says 172.21.15.133:33336) 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.747+0000 7fbc85000700 1 -- 172.21.15.133:0/1907689350 learned_addr learned my addr 172.21.15.133:0/1907689350 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.747+0000 7fbc85000700 1 -- 172.21.15.133:0/1907689350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8019e360 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.747+0000 7fbc85000700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011b60 tx=0x7fbc74011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.748+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74003b20 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.748+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc74003cc0 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.748+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74024840 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.748+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8019e5f0 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.748+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc801a12e0 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.749+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fbc74022020 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.749+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc80063980 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.749+0000 7fbc7d600700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 0x7fbc6803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.750+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc74011e00 con 0x7fbc8007de10 2024-05-09T08:19:15.702 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.750+0000 7fbc7fe00700 1 -- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 msgr2=0x7fbc6803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.750+0000 7fbc7fe00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 0x7fbc6803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.755+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc7400ba30 con 0x7fbc8007de10 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.951+0000 7fbc7fe00700 1 -- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 msgr2=0x7fbc6803b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:14.951+0000 7fbc7fe00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 0x7fbc6803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.034+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbc801a14b0 con 0x7fbc8007de10 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.035+0000 7fbc7d600700 1 -- 172.21.15.133:0/1907689350 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fbc801a14b0 con 0x7fbc8007de10 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.037+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 msgr2=0x7fbc6803b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.037+0000 7fbc87a00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 0x7fbc6803b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.037+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 msgr2=0x7fbc8019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.037+0000 7fbc87a00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011b60 tx=0x7fbc74011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 shutdown_connections 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7fbc68039160 0x7fbc6803b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 --2- 172.21.15.133:0/1907689350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8007de10 0x7fbc8019de20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:15.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 >> 172.21.15.133:0/1907689350 conn(0x7fbc801032e0 msgr2=0x7fbc80103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:15.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 shutdown_connections 2024-05-09T08:19:15.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:15.038+0000 7fbc87a00700 1 -- 172.21.15.133:0/1907689350 wait complete. 2024-05-09T08:19:15.704 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2024-05-09T08:19:15.704 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 5... 2024-05-09T08:19:16.468 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:16.214+0000 7fe39111e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:19:16.469 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:16.443+0000 7fe39111e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:16.870 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:16.657+0000 7fe39111e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:19:17.157 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:17 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:17.035+0000 7fe39111e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:19:17.158 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:17 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:17.154+0000 7fe39111e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:19:17.671 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:17 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:17.416+0000 7fe39111e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:19:17.925 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:17 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:17.679+0000 7fe39111e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:19:18.852 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:18.499+0000 7fe39111e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:19:19.180 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:19 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:19.059+0000 7fe39111e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:19:19.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:19 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:19.186+0000 7fe39111e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:19:19.454 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:19 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:19.451+0000 7fe39111e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:19:20.429 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.092+0000 7fe39111e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:19:20.430 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.215+0000 7fe39111e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:19:20.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.532+0000 7fe39111e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:19:20.840 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.710+0000 7fe39111e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:19:20.840 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.836+0000 7fe39111e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:19:21.179 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:20.988+0000 7fe39111e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:21.679 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:21 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:21.359+0000 7fe39111e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:19:22.398 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:22 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:22.113+0000 7fe39111e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:22 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:22.395+0000 7fe39111e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: Active manager daemon x restarted 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: Activating manager daemon x 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: mgrmap e6: x(active, starting, since 0.00968999s) 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:19:22.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:19:22.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:19:22.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:19:22.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:19:22.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: Manager daemon x is now available 2024-05-09T08:19:22.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:22 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: Found migration_current of "None". Setting to last migration. 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:19:23.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:23 smithi133 ceph-mon[29610]: mgrmap e7: x(active, since 1.01734s) 2024-05-09T08:19:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T08:19:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T08:19:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:24.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.257+0000 7f0e95000700 1 Processor -- start 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.257+0000 7f0e95000700 1 -- start start 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.258+0000 7f0e95000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.258+0000 7f0e95000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9007c820 con 0x7f0e9007dde0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.258+0000 7f0e8de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.258+0000 7f0e8de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33346/0 (socket says 172.21.15.133:33346) 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.258+0000 7f0e8de00700 1 -- 172.21.15.133:0/1351602876 learned_addr learned my addr 172.21.15.133:0/1351602876 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.259+0000 7f0e8de00700 1 -- 172.21.15.133:0/1351602876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9007c9a0 con 0x7f0e9007dde0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.259+0000 7f0e8de00700 1 --2- 172.21.15.133:0/1351602876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098a0 tx=0x7f0e80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3310fffa761c779 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.259+0000 7f0e8ca00700 1 -- 172.21.15.133:0/1351602876 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e80004160 con 0x7f0e9007dde0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.259+0000 7f0e8ca00700 1 -- 172.21.15.133:0/1351602876 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e80004300 con 0x7f0e9007dde0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.259+0000 7f0e8ca00700 1 -- 172.21.15.133:0/1351602876 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e80011b50 con 0x7f0e9007dde0 2024-05-09T08:19:24.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 -- 172.21.15.133:0/1351602876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 msgr2=0x7f0e9007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 --2- 172.21.15.133:0/1351602876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098a0 tx=0x7f0e80009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 -- 172.21.15.133:0/1351602876 shutdown_connections 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 --2- 172.21.15.133:0/1351602876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9007c2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 -- 172.21.15.133:0/1351602876 >> 172.21.15.133:0/1351602876 conn(0x7f0e90103320 msgr2=0x7f0e90105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 -- 172.21.15.133:0/1351602876 shutdown_connections 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.260+0000 7f0e95000700 1 -- 172.21.15.133:0/1351602876 wait complete. 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.261+0000 7f0e95000700 1 Processor -- start 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e95000700 1 -- start start 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e95000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e95000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9010e310 con 0x7f0e9007dde0 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e8de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e8de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33360/0 (socket says 172.21.15.133:33360) 2024-05-09T08:19:24.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.262+0000 7f0e8de00700 1 -- 172.21.15.133:0/2193094132 learned_addr learned my addr 172.21.15.133:0/2193094132 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e8de00700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9019e3a0 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e8de00700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e80011b60 tx=0x7f0e80011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e80003970 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e80003b10 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9019e630 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e80024610 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.263+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a1320 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0e8001a9f0 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e86a00700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e7403bce0 con 0x7f0e74039110 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e80011e00 con 0x7f0e9007dde0 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e8d400700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:24.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.265+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.466+0000 7f0e8d400700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.466+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.867+0000 7f0e8d400700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:16.867+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:17.668+0000 7f0e8d400700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:17.668+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:19.270+0000 7f0e8d400700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/1186224387 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:19.270+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:22.405+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f0e80024c90 con 0x7f0e9007dde0 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:22.405+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 msgr2=0x7f0e7403b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:22.405+0000 7f0e86a00700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.410+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f0e8004eee0 con 0x7f0e9007dde0 2024-05-09T08:19:24.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.410+0000 7f0e86a00700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 0x7f0e7403ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.410+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e8000ce90 con 0x7f0e7403ca40 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.413+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 0x7f0e7403ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.415+0000 7f0e8d400700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 0x7f0e7403ee30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003a10 tx=0x7f0e7c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.416+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0e8000ce90 con 0x7f0e7403ca40 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.425+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0e90069440 con 0x7f0e7403ca40 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.425+0000 7f0e86a00700 1 -- 172.21.15.133:0/2193094132 <== mgr.14120 v2:172.21.15.133:6800/3643547543 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0e90069440 con 0x7f0e7403ca40 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 msgr2=0x7f0e7403ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 0x7f0e7403ee30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003a10 tx=0x7f0e7c009290 comp rx=0 tx=0).stop 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 msgr2=0x7f0e9019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e80011b60 tx=0x7f0e80011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 shutdown_connections 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f0e7403ca40 0x7f0e7403ee30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.154 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e9007dde0 0x7f0e9019de60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 --2- 172.21.15.133:0/2193094132 >> [v2:172.21.15.133:6800/1186224387,v1:172.21.15.133:6801/1186224387] conn(0x7f0e74039110 0x7f0e7403b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 >> 172.21.15.133:0/2193094132 conn(0x7f0e90103320 msgr2=0x7f0e90103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 shutdown_connections 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:23.426+0000 7f0e95000700 1 -- 172.21.15.133:0/2193094132 wait complete. 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 5 is available 2024-05-09T08:19:24.155 INFO:teuthology.orchestra.run.smithi133.stdout:Setting orchestrator backend to cephadm... 2024-05-09T08:19:24.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:24 smithi133 ceph-mon[29610]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T08:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:24 smithi133 ceph-mon[29610]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T08:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:24 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:24 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:24 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: [09/May/2024:08:19:24] ENGINE Bus STARTING 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: [09/May/2024:08:19:24] ENGINE Serving on https://172.21.15.133:7150 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: [09/May/2024:08:19:24] ENGINE Serving on http://172.21.15.133:8765 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: [09/May/2024:08:19:24] ENGINE Bus STARTED 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: mgrmap e8: x(active, since 2s) 2024-05-09T08:19:25.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:25.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:25 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.774+0000 7f7fed800700 1 Processor -- start 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.774+0000 7f7fed800700 1 -- start start 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.774+0000 7f7fed800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.775+0000 7f7fed800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810a8c0 con 0x7f7fe8109f60 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.775+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.775+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36564/0 (socket says 172.21.15.133:36564) 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.775+0000 7f7fe6a00700 1 -- 172.21.15.133:0/3671564288 learned_addr learned my addr 172.21.15.133:0/3671564288 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:25.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.776+0000 7f7fe6a00700 1 -- 172.21.15.133:0/3671564288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe810b070 con 0x7f7fe8109f60 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.776+0000 7f7fe6a00700 1 --2- 172.21.15.133:0/3671564288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80098a0 tx=0x7f7fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66f85b9adae52a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fe5600700 1 -- 172.21.15.133:0/3671564288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fd8004160 con 0x7f7fe8109f60 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fe5600700 1 -- 172.21.15.133:0/3671564288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fd8004300 con 0x7f7fe8109f60 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fe5600700 1 -- 172.21.15.133:0/3671564288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fd8011b50 con 0x7f7fe8109f60 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fed800700 1 -- 172.21.15.133:0/3671564288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 msgr2=0x7f7fe810a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fed800700 1 --2- 172.21.15.133:0/3671564288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80098a0 tx=0x7f7fd8009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fed800700 1 -- 172.21.15.133:0/3671564288 shutdown_connections 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.777+0000 7f7fed800700 1 --2- 172.21.15.133:0/3671564288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe810a380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.778+0000 7f7fed800700 1 -- 172.21.15.133:0/3671564288 >> 172.21.15.133:0/3671564288 conn(0x7f7fe807d2e0 msgr2=0x7f7fe807d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.778+0000 7f7fed800700 1 -- 172.21.15.133:0/3671564288 shutdown_connections 2024-05-09T08:19:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.778+0000 7f7fed800700 1 -- 172.21.15.133:0/3671564288 wait complete. 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.779+0000 7f7fed800700 1 Processor -- start 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.779+0000 7f7fed800700 1 -- start start 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.779+0000 7f7fed800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.779+0000 7f7fed800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810e4b0 con 0x7f7fe8109f60 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.779+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.780+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36570/0 (socket says 172.21.15.133:36570) 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.780+0000 7f7fe6a00700 1 -- 172.21.15.133:0/1132157253 learned_addr learned my addr 172.21.15.133:0/1132157253 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.780+0000 7f7fe6a00700 1 -- 172.21.15.133:0/1132157253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe819e2f0 con 0x7f7fe8109f60 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.780+0000 7f7fe6a00700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011670 tx=0x7f7fd8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.783+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fd80039c0 con 0x7f7fe8109f60 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.783+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe819e580 con 0x7f7fe8109f60 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.783+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fd8011e50 con 0x7f7fe8109f60 2024-05-09T08:19:25.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.783+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fd801a5c0 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.783+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81a1230 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.784+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7fd801a870 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.784+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe8063980 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.784+0000 7f7fdf400700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 0x7f7fcc03b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.784+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7fd804d510 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.785+0000 7f7fe6000700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 0x7f7fcc03b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.785+0000 7f7fe6000700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 0x7f7fcc03b5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:24.791+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7fd802cc00 con 0x7f7fe8109f60 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.033+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7fe8107aa0 con 0x7f7fcc039110 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.047+0000 7f7fdf400700 1 -- 172.21.15.133:0/1132157253 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7fe8107aa0 con 0x7f7fcc039110 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 msgr2=0x7f7fcc03b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:25.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 0x7f7fcc03b5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006fd0 tx=0x7f7fd4006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 msgr2=0x7f7fe819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8011670 tx=0x7f7fd8011b60 comp rx=0 tx=0).stop 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 shutdown_connections 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f7fcc039110 0x7f7fcc03b5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 --2- 172.21.15.133:0/1132157253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8109f60 0x7f7fe819ddb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.051+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 >> 172.21.15.133:0/1132157253 conn(0x7f7fe807d2e0 msgr2=0x7f7fe8107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.052+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 shutdown_connections 2024-05-09T08:19:25.814 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:25.052+0000 7f7fed800700 1 -- 172.21.15.133:0/1132157253 wait complete. 2024-05-09T08:19:26.642 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:26 smithi133 ceph-mon[29610]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:27.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T08:19:27.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.375+0000 7f027ca00700 1 Processor -- start 2024-05-09T08:19:27.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.375+0000 7f027ca00700 1 -- start start 2024-05-09T08:19:27.331 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.376+0000 7f027ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:27.332 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.376+0000 7f027ca00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0278111600 con 0x7f027810ecd0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.377+0000 7f0275e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.377+0000 7f0275e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36580/0 (socket says 172.21.15.133:36580) 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.377+0000 7f0275e00700 1 -- 172.21.15.133:0/2797494972 learned_addr learned my addr 172.21.15.133:0/2797494972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.377+0000 7f0275e00700 1 -- 172.21.15.133:0/2797494972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0278111780 con 0x7f027810ecd0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.378+0000 7f0275e00700 1 --2- 172.21.15.133:0/2797494972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02680098a0 tx=0x7f0268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60309c4eae40df5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.378+0000 7f0274a00700 1 -- 172.21.15.133:0/2797494972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0268004160 con 0x7f027810ecd0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.378+0000 7f0274a00700 1 -- 172.21.15.133:0/2797494972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0268004300 con 0x7f027810ecd0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.378+0000 7f0274a00700 1 -- 172.21.15.133:0/2797494972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0268011b50 con 0x7f027810ecd0 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 -- 172.21.15.133:0/2797494972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 msgr2=0x7f02781110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 --2- 172.21.15.133:0/2797494972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02680098a0 tx=0x7f0268009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:27.338 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 -- 172.21.15.133:0/2797494972 shutdown_connections 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 --2- 172.21.15.133:0/2797494972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781110c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 -- 172.21.15.133:0/2797494972 >> 172.21.15.133:0/2797494972 conn(0x7f027807d380 msgr2=0x7f027807f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 -- 172.21.15.133:0/2797494972 shutdown_connections 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.379+0000 7f027ca00700 1 -- 172.21.15.133:0/2797494972 wait complete. 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.380+0000 7f027ca00700 1 Processor -- start 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.380+0000 7f027ca00700 1 -- start start 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.380+0000 7f027ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.380+0000 7f027ca00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0278112730 con 0x7f027810ecd0 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.381+0000 7f0275e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.381+0000 7f0275e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36584/0 (socket says 172.21.15.133:36584) 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.381+0000 7f0275e00700 1 -- 172.21.15.133:0/2841047057 learned_addr learned my addr 172.21.15.133:0/2841047057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.381+0000 7f0275e00700 1 -- 172.21.15.133:0/2841047057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02781a2740 con 0x7f027810ecd0 2024-05-09T08:19:27.339 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.381+0000 7f0275e00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0268011b60 tx=0x7f0268011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.382+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0268003970 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.382+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0268003b10 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.382+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0268024610 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.382+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02781a29d0 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.382+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02781a56c0 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.383+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f026801a9f0 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.383+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0278069440 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.384+0000 7f026ea00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 0x7f025c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.384+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0268011e00 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.384+0000 7f0275400700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 0x7f025c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.385+0000 7f0275400700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 0x7f025c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.390+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f026800b3e0 con 0x7f027810ecd0 2024-05-09T08:19:27.340 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.616+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f027807e510 con 0x7f025c039160 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.618+0000 7f026ea00700 1 -- 172.21.15.133:0/2841047057 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f027807e510 con 0x7f025c039160 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 msgr2=0x7f025c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 0x7f025c03b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0264006fd0 tx=0x7f0264006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 msgr2=0x7f02781a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0268011b60 tx=0x7f0268011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 shutdown_connections 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f025c039160 0x7f025c03b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 --2- 172.21.15.133:0/2841047057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f027810ecd0 0x7f02781a2200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 >> 172.21.15.133:0/2841047057 conn(0x7f027807d380 msgr2=0x7f027807de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 shutdown_connections 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:26.622+0000 7f027ca00700 1 -- 172.21.15.133:0/2841047057 wait complete. 2024-05-09T08:19:27.341 INFO:teuthology.orchestra.run.smithi133.stdout:Generating ssh key... 2024-05-09T08:19:27.904 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:27 smithi133 ceph-mon[29610]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:28.838 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: Generating public/private rsa key pair. 2024-05-09T08:19:28.838 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: Your identification has been saved in /tmp/tmp8_beezxx/key. 2024-05-09T08:19:28.838 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: Your public key has been saved in /tmp/tmp8_beezxx/key.pub. 2024-05-09T08:19:28.838 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: The key fingerprint is: 2024-05-09T08:19:28.838 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: SHA256:PvCusaUFZrHHIYuIkja/ObwqvaFWC8tWnVy/+WjJns4 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: The key's randomart image is: 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: +---[RSA 3072]----+ 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: | | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: | | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: | o . | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: | o . . B . | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: |+o. + X S | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: |o.oo * * . | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: |..B.. . B + | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: |.*.=o B.Bo | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: |=.o=o +.=E.. | 2024-05-09T08:19:28.839 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: +----[SHA256]-----+ 2024-05-09T08:19:29.149 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.939+0000 7f291a000700 1 Processor -- start 2024-05-09T08:19:29.150 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.939+0000 7f291a000700 1 -- start start 2024-05-09T08:19:29.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.940+0000 7f291a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:29.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.940+0000 7f291a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f291407c8f0 con 0x7f291407dec0 2024-05-09T08:19:29.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.940+0000 7f2918c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:29.151 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.940+0000 7f2918c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36598/0 (socket says 172.21.15.133:36598) 2024-05-09T08:19:29.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.940+0000 7f2918c00700 1 -- 172.21.15.133:0/2716899210 learned_addr learned my addr 172.21.15.133:0/2716899210 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:29.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.941+0000 7f2918c00700 1 -- 172.21.15.133:0/2716899210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291407ca70 con 0x7f291407dec0 2024-05-09T08:19:29.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.941+0000 7f2918c00700 1 --2- 172.21.15.133:0/2716899210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2908009e80 tx=0x7f290800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e15f32c4dd5b63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:29.152 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.941+0000 7f2913400700 1 -- 172.21.15.133:0/2716899210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2908017070 con 0x7f291407dec0 2024-05-09T08:19:29.153 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.941+0000 7f2913400700 1 -- 172.21.15.133:0/2716899210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f290800b8c0 con 0x7f291407dec0 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.941+0000 7f2913400700 1 -- 172.21.15.133:0/2716899210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29080045c0 con 0x7f291407dec0 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 -- 172.21.15.133:0/2716899210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 msgr2=0x7f291407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 --2- 172.21.15.133:0/2716899210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2908009e80 tx=0x7f290800b330 comp rx=0 tx=0).stop 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 -- 172.21.15.133:0/2716899210 shutdown_connections 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 --2- 172.21.15.133:0/2716899210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291407c320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:29.158 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 -- 172.21.15.133:0/2716899210 >> 172.21.15.133:0/2716899210 conn(0x7f2914103170 msgr2=0x7f2914105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 -- 172.21.15.133:0/2716899210 shutdown_connections 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.942+0000 7f291a000700 1 -- 172.21.15.133:0/2716899210 wait complete. 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.943+0000 7f291a000700 1 Processor -- start 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.943+0000 7f291a000700 1 -- start start 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f291a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f291a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f291410e180 con 0x7f291407dec0 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f2918c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f2918c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36606/0 (socket says 172.21.15.133:36606) 2024-05-09T08:19:29.159 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f2918c00700 1 -- 172.21.15.133:0/928294042 learned_addr learned my addr 172.21.15.133:0/928294042 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f2918c00700 1 -- 172.21.15.133:0/928294042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291419e1c0 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.944+0000 7f2918c00700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2908004210 tx=0x7f290800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.945+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2908020020 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.945+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2908003b50 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.945+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2908017040 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.945+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291419e450 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.945+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141a1140 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.946+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2914069440 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.946+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2908023020 con 0x7f291407dec0 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.947+0000 7f2911600700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 0x7f28fc03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:29.160 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.947+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f290800f050 con 0x7f291407dec0 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.947+0000 7f2913e00700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 0x7f28fc03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.947+0000 7f2913e00700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 0x7f28fc03b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2900006fd0 tx=0x7f2900006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:27.952+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f290802a020 con 0x7f291407dec0 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.172+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2914104320 con 0x7f28fc039160 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.469+0000 7f2911600700 1 -- 172.21.15.133:0/928294042 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2914104320 con 0x7f28fc039160 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.473+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 msgr2=0x7f28fc03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.473+0000 7f291a000700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 0x7f28fc03b620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2900006fd0 tx=0x7f2900006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.473+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 msgr2=0x7f291419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.473+0000 7f291a000700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2908004210 tx=0x7f290800bd50 comp rx=0 tx=0).stop 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 shutdown_connections 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f28fc039160 0x7f28fc03b620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:29.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 --2- 172.21.15.133:0/928294042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f291407dec0 0x7f291419dc80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:29.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 >> 172.21.15.133:0/928294042 conn(0x7f2914103170 msgr2=0x7f2914103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:29.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 shutdown_connections 2024-05-09T08:19:29.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:28.474+0000 7f291a000700 1 -- 172.21.15.133:0/928294042 wait complete. 2024-05-09T08:19:29.743 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:29 smithi133 ceph-mon[29610]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:29.743 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:29 smithi133 ceph-mon[29610]: Generating ssh key... 2024-05-09T08:19:29.743 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:29 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:29.743 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:29 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:30.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:30 smithi133 ceph-mon[29610]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:30.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:19:30.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.805+0000 7f5241200700 1 Processor -- start 2024-05-09T08:19:30.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.806+0000 7f5241200700 1 -- start start 2024-05-09T08:19:30.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.806+0000 7f5241200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:30.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.806+0000 7f5241200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f523c111600 con 0x7f523c10ecd0 2024-05-09T08:19:30.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.806+0000 7f5239e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:30.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.807+0000 7f5239e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36610/0 (socket says 172.21.15.133:36610) 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.807+0000 7f5239e00700 1 -- 172.21.15.133:0/957112041 learned_addr learned my addr 172.21.15.133:0/957112041 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.807+0000 7f5239e00700 1 -- 172.21.15.133:0/957112041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c111780 con 0x7f523c10ecd0 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.807+0000 7f5239e00700 1 --2- 172.21.15.133:0/957112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f522c0098a0 tx=0x7f522c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf34056d8409147 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.808+0000 7f5238a00700 1 -- 172.21.15.133:0/957112041 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c004160 con 0x7f523c10ecd0 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.808+0000 7f5238a00700 1 -- 172.21.15.133:0/957112041 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f522c004300 con 0x7f523c10ecd0 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.808+0000 7f5238a00700 1 -- 172.21.15.133:0/957112041 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c011b50 con 0x7f523c10ecd0 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.808+0000 7f5241200700 1 -- 172.21.15.133:0/957112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 msgr2=0x7f523c1110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:30.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.808+0000 7f5241200700 1 --2- 172.21.15.133:0/957112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f522c0098a0 tx=0x7f522c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.809+0000 7f5241200700 1 -- 172.21.15.133:0/957112041 shutdown_connections 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.809+0000 7f5241200700 1 --2- 172.21.15.133:0/957112041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1110c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.809+0000 7f5241200700 1 -- 172.21.15.133:0/957112041 >> 172.21.15.133:0/957112041 conn(0x7f523c07d2d0 msgr2=0x7f523c07f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.809+0000 7f5241200700 1 -- 172.21.15.133:0/957112041 shutdown_connections 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.809+0000 7f5241200700 1 -- 172.21.15.133:0/957112041 wait complete. 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.810+0000 7f5241200700 1 Processor -- start 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.810+0000 7f5241200700 1 -- start start 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.810+0000 7f5241200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.810+0000 7f5241200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1126f0 con 0x7f523c10ecd0 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.811+0000 7f5239e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.811+0000 7f5239e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36614/0 (socket says 172.21.15.133:36614) 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.811+0000 7f5239e00700 1 -- 172.21.15.133:0/1299958207 learned_addr learned my addr 172.21.15.133:0/1299958207 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.811+0000 7f5239e00700 1 -- 172.21.15.133:0/1299958207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1a26c0 con 0x7f523c10ecd0 2024-05-09T08:19:30.789 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.811+0000 7f5239e00700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f522c011b60 tx=0x7f522c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.812+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c003b20 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.812+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f522c003cc0 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.812+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f522c024840 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.812+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1a2950 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.812+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a5640 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.813+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f522c022020 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.813+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c063980 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.813+0000 7f5232a00700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 0x7f522003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.813+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f522c011e00 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.814+0000 7f5239400700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 0x7f522003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.814+0000 7f5239400700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 0x7f522003b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5228006fd0 tx=0x7f5228006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:29.819+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f522c00b3e0 con 0x7f523c10ecd0 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.042+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f523c07e3e0 con 0x7f5220039160 2024-05-09T08:19:30.790 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.043+0000 7f5232a00700 1 -- 172.21.15.133:0/1299958207 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f523c07e3e0 con 0x7f5220039160 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 msgr2=0x7f522003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 0x7f522003b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5228006fd0 tx=0x7f5228006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 msgr2=0x7f523c1a2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f522c011b60 tx=0x7f522c011c40 comp rx=0 tx=0).stop 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 shutdown_connections 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f5220039160 0x7f522003b620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 --2- 172.21.15.133:0/1299958207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523c10ecd0 0x7f523c1a2180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 >> 172.21.15.133:0/1299958207 conn(0x7f523c07d2d0 msgr2=0x7f523c07dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 shutdown_connections 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:30.047+0000 7f5241200700 1 -- 172.21.15.133:0/1299958207 wait complete. 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T08:19:30.791 INFO:teuthology.orchestra.run.smithi133.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T08:19:30.792 INFO:teuthology.orchestra.run.smithi133.stdout:Adding host smithi133... 2024-05-09T08:19:32.820 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:32 smithi133 ceph-mon[29610]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:33 smithi133 ceph-mon[29610]: Deploying cephadm binary to smithi133 2024-05-09T08:19:38.189 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Added host 'smithi133' with addr '172.21.15.133' 2024-05-09T08:19:38.189 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.372+0000 7fd643c00700 1 Processor -- start 2024-05-09T08:19:38.189 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.372+0000 7fd643c00700 1 -- start start 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.372+0000 7fd643c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.372+0000 7fd643c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c108710 con 0x7fd63c107d20 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.373+0000 7fd641200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.373+0000 7fd641200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36620/0 (socket says 172.21.15.133:36620) 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.373+0000 7fd641200700 1 -- 172.21.15.133:0/3991390119 learned_addr learned my addr 172.21.15.133:0/3991390119 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:38.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.373+0000 7fd641200700 1 -- 172.21.15.133:0/3991390119 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c108f20 con 0x7fd63c107d20 2024-05-09T08:19:38.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.373+0000 7fd641200700 1 --2- 172.21.15.133:0/3991390119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098a0 tx=0x7fd630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc895fed6fa2fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:38.196 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd63b400700 1 -- 172.21.15.133:0/3991390119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630004160 con 0x7fd63c107d20 2024-05-09T08:19:38.196 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd63b400700 1 -- 172.21.15.133:0/3991390119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd630004300 con 0x7fd63c107d20 2024-05-09T08:19:38.196 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd63b400700 1 -- 172.21.15.133:0/3991390119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630011b50 con 0x7fd63c107d20 2024-05-09T08:19:38.196 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd643c00700 1 -- 172.21.15.133:0/3991390119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 msgr2=0x7fd63c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:38.196 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd643c00700 1 --2- 172.21.15.133:0/3991390119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6300098a0 tx=0x7fd630009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd643c00700 1 -- 172.21.15.133:0/3991390119 shutdown_connections 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd643c00700 1 --2- 172.21.15.133:0/3991390119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c108140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.374+0000 7fd643c00700 1 -- 172.21.15.133:0/3991390119 >> 172.21.15.133:0/3991390119 conn(0x7fd63c1032e0 msgr2=0x7fd63c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.375+0000 7fd643c00700 1 -- 172.21.15.133:0/3991390119 shutdown_connections 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.375+0000 7fd643c00700 1 -- 172.21.15.133:0/3991390119 wait complete. 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.376+0000 7fd643c00700 1 Processor -- start 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.376+0000 7fd643c00700 1 -- start start 2024-05-09T08:19:38.197 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.376+0000 7fd643c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.376+0000 7fd643c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10c2d0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.377+0000 7fd641200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.377+0000 7fd641200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36636/0 (socket says 172.21.15.133:36636) 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.377+0000 7fd641200700 1 -- 172.21.15.133:0/1157466160 learned_addr learned my addr 172.21.15.133:0/1157466160 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.377+0000 7fd641200700 1 -- 172.21.15.133:0/1157466160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c07e0b0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.377+0000 7fd641200700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd630011850 tx=0x7fd630011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.378+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd630003af0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.378+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd630020070 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.378+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c07c2a0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.378+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6300195a0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.378+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c07c7e0 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.379+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd630019850 con 0x7fd63c107d20 2024-05-09T08:19:38.198 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.379+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c063980 con 0x7fd63c107d20 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.379+0000 7fd639600700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 0x7fd62403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.380+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd630011e00 con 0x7fd63c107d20 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.380+0000 7fd63be00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 0x7fd62403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.380+0000 7fd63be00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 0x7fd62403b680 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd628006fd0 tx=0x7fd628006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.385+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd630003c90 con 0x7fd63c107d20 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:31.609+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}) -- 0x7fd63c104440 con 0x7fd6240391c0 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.474+0000 7fd639600700 1 -- 172.21.15.133:0/1157466160 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd63c104440 con 0x7fd6240391c0 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 msgr2=0x7fd62403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 0x7fd62403b680 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd628006fd0 tx=0x7fd628006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 msgr2=0x7fd63c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd630011850 tx=0x7fd630011c10 comp rx=0 tx=0).stop 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 shutdown_connections 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd6240391c0 0x7fd62403b680 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:38.199 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 --2- 172.21.15.133:0/1157466160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd63c107d20 0x7fd63c07db70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:38.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.478+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 >> 172.21.15.133:0/1157466160 conn(0x7fd63c1032e0 msgr2=0x7fd63c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:38.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.479+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 shutdown_connections 2024-05-09T08:19:38.200 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:37.479+0000 7fd643c00700 1 -- 172.21.15.133:0/1157466160 wait complete. 2024-05-09T08:19:38.200 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mon service... 2024-05-09T08:19:38.726 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:38 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:38.726 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:38 smithi133 ceph-mon[29610]: Added host smithi133 2024-05-09T08:19:38.726 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:38 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:40.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T08:19:40.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.891+0000 7f3f1de00700 1 Processor -- start 2024-05-09T08:19:40.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.891+0000 7f3f1de00700 1 -- start start 2024-05-09T08:19:40.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.892+0000 7f3f1de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:40.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.892+0000 7f3f1de00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1807c890 con 0x7f3f1807de60 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.892+0000 7f3f1ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.892+0000 7f3f1ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40924/0 (socket says 172.21.15.133:40924) 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.892+0000 7f3f1ca00700 1 -- 172.21.15.133:0/2390488360 learned_addr learned my addr 172.21.15.133:0/2390488360 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.893+0000 7f3f1ca00700 1 -- 172.21.15.133:0/2390488360 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1807ca10 con 0x7f3f1807de60 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.893+0000 7f3f1ca00700 1 --2- 172.21.15.133:0/2390488360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e80 tx=0x7f3f0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db187a14db8f7a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.893+0000 7f3f17400700 1 -- 172.21.15.133:0/2390488360 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f0c017070 con 0x7f3f1807de60 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f17400700 1 -- 172.21.15.133:0/2390488360 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f0c00b8c0 con 0x7f3f1807de60 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f17400700 1 -- 172.21.15.133:0/2390488360 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f0c0045c0 con 0x7f3f1807de60 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 -- 172.21.15.133:0/2390488360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 msgr2=0x7f3f1807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 --2- 172.21.15.133:0/2390488360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e80 tx=0x7f3f0c00b330 comp rx=0 tx=0).stop 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 -- 172.21.15.133:0/2390488360 shutdown_connections 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 --2- 172.21.15.133:0/2390488360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1807c2c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 -- 172.21.15.133:0/2390488360 >> 172.21.15.133:0/2390488360 conn(0x7f3f18103170 msgr2=0x7f3f18105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:40.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.894+0000 7f3f1de00700 1 -- 172.21.15.133:0/2390488360 shutdown_connections 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.895+0000 7f3f1de00700 1 -- 172.21.15.133:0/2390488360 wait complete. 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1de00700 1 Processor -- start 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1de00700 1 -- start start 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1de00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1810e180 con 0x7f3f1807de60 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:40.034 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1ca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40930/0 (socket says 172.21.15.133:40930) 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.896+0000 7f3f1ca00700 1 -- 172.21.15.133:0/417600995 learned_addr learned my addr 172.21.15.133:0/417600995 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f1ca00700 1 -- 172.21.15.133:0/417600995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1819e1a0 con 0x7f3f1807de60 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f1ca00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e50 tx=0x7f3f0c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f0c020020 con 0x7f3f1807de60 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3f0c003b50 con 0x7f3f1807de60 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1819e430 con 0x7f3f1807de60 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.897+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f0c017040 con 0x7f3f1807de60 2024-05-09T08:19:40.037 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.898+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181a1120 con 0x7f3f1807de60 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.898+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f18069440 con 0x7f3f1807de60 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.899+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3f0c023020 con 0x7f3f1807de60 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.899+0000 7f3f15600700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 0x7f3f0003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.899+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f0c00f050 con 0x7f3f1807de60 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.899+0000 7f3f17e00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 0x7f3f0003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.900+0000 7f3f17e00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 0x7f3f0003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006fd0 tx=0x7f3f04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:38.905+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f0c02a020 con 0x7f3f1807de60 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.146+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3f18104300 con 0x7f3f000390c0 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.158+0000 7f3f15600700 1 -- 172.21.15.133:0/417600995 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3f18104300 con 0x7f3f000390c0 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.160+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 msgr2=0x7f3f0003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.160+0000 7f3f1de00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 0x7f3f0003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006fd0 tx=0x7f3f04006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.160+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 msgr2=0x7f3f1819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.160+0000 7f3f1de00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e50 tx=0x7f3f0c00bd30 comp rx=0 tx=0).stop 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 shutdown_connections 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f3f000390c0 0x7f3f0003b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 --2- 172.21.15.133:0/417600995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f1807de60 0x7f3f1819dc60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 >> 172.21.15.133:0/417600995 conn(0x7f3f18103170 msgr2=0x7f3f18103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 shutdown_connections 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:39.161+0000 7f3f1de00700 1 -- 172.21.15.133:0/417600995 wait complete. 2024-05-09T08:19:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mgr service... 2024-05-09T08:19:40.341 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:40 smithi133 ceph-mon[29610]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:40.341 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:40 smithi133 ceph-mon[29610]: Saving service mon spec with placement count:5 2024-05-09T08:19:40.341 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:40 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:42.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:42 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:42.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:42 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:42.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:42 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:42.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T08:19:42.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.574+0000 7f85b7400700 1 Processor -- start 2024-05-09T08:19:42.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.574+0000 7f85b7400700 1 -- start start 2024-05-09T08:19:42.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.574+0000 7f85b7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:42.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.574+0000 7f85b7400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85b007be50 con 0x7f85b01087f0 2024-05-09T08:19:42.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.575+0000 7f85b4a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:42.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.575+0000 7f85b4a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40932/0 (socket says 172.21.15.133:40932) 2024-05-09T08:19:42.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.575+0000 7f85b4a00700 1 -- 172.21.15.133:0/184856555 learned_addr learned my addr 172.21.15.133:0/184856555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:42.516 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.575+0000 7f85b4a00700 1 -- 172.21.15.133:0/184856555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b010b120 con 0x7f85b01087f0 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.576+0000 7f85b4a00700 1 --2- 172.21.15.133:0/184856555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85a40098a0 tx=0x7f85a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161fc125a5045daf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.576+0000 7f85af400700 1 -- 172.21.15.133:0/184856555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85a4004160 con 0x7f85b01087f0 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.576+0000 7f85af400700 1 -- 172.21.15.133:0/184856555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a4004300 con 0x7f85b01087f0 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.576+0000 7f85af400700 1 -- 172.21.15.133:0/184856555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85a4011b20 con 0x7f85b01087f0 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 -- 172.21.15.133:0/184856555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 msgr2=0x7f85b010abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 --2- 172.21.15.133:0/184856555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85a40098a0 tx=0x7f85a4009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 -- 172.21.15.133:0/184856555 shutdown_connections 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 --2- 172.21.15.133:0/184856555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b010abe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 -- 172.21.15.133:0/184856555 >> 172.21.15.133:0/184856555 conn(0x7f85b01021e0 msgr2=0x7f85b0104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 -- 172.21.15.133:0/184856555 shutdown_connections 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.577+0000 7f85b7400700 1 -- 172.21.15.133:0/184856555 wait complete. 2024-05-09T08:19:42.518 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.578+0000 7f85b7400700 1 Processor -- start 2024-05-09T08:19:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.578+0000 7f85b7400700 1 -- start start 2024-05-09T08:19:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b7400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010c130 con 0x7f85b01087f0 2024-05-09T08:19:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b4a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:42.519 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b4a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40938/0 (socket says 172.21.15.133:40938) 2024-05-09T08:19:42.520 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b4a00700 1 -- 172.21.15.133:0/2574632387 learned_addr learned my addr 172.21.15.133:0/2574632387 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:42.520 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.579+0000 7f85b4a00700 1 -- 172.21.15.133:0/2574632387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b019e2f0 con 0x7f85b01087f0 2024-05-09T08:19:42.520 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85b4a00700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85a40117e0 tx=0x7f85a4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85a4003ba0 con 0x7f85b01087f0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a401a910 con 0x7f85b01087f0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85a40248f0 con 0x7f85b01087f0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b019e580 con 0x7f85b01087f0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.580+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b01a1230 con 0x7f85b01087f0 2024-05-09T08:19:42.521 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.581+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0063980 con 0x7f85b01087f0 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.582+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85a4022020 con 0x7f85b01087f0 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.582+0000 7f85ad600700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 0x7f859803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.582+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85a404d580 con 0x7f85b01087f0 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.583+0000 7f85afe00700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 0x7f859803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.584+0000 7f85afe00700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 0x7f859803b620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f859c006fd0 tx=0x7f859c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:42.522 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.588+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85a400f2e0 con 0x7f85b01087f0 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.840+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f85b01032f0 con 0x7f8598039160 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.847+0000 7f85ad600700 1 -- 172.21.15.133:0/2574632387 <== mgr.14120 v2:172.21.15.133:6800/3643547543 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f85b01032f0 con 0x7f8598039160 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 msgr2=0x7f859803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 0x7f859803b620 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f859c006fd0 tx=0x7f859c006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 msgr2=0x7f85b019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:42.523 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85a40117e0 tx=0x7f85a4011ba0 comp rx=0 tx=0).stop 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 shutdown_connections 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f8598039160 0x7f859803b620 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 --2- 172.21.15.133:0/2574632387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01087f0 0x7f85b019ddb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 >> 172.21.15.133:0/2574632387 conn(0x7f85b01021e0 msgr2=0x7f85b0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 shutdown_connections 2024-05-09T08:19:42.524 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:41.850+0000 7f85b7400700 1 -- 172.21.15.133:0/2574632387 wait complete. 2024-05-09T08:19:43.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:43 smithi133 ceph-mon[29610]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:19:43.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:43 smithi133 ceph-mon[29610]: Saving service mgr spec with placement count:2 2024-05-09T08:19:43.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:43 smithi133 ceph-mon[29610]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:19:43.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:43 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/61957323' entity='client.admin' 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.122+0000 7f7581400700 1 Processor -- start 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.122+0000 7f7581400700 1 -- start start 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.122+0000 7f7581400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.122+0000 7f7581400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f757c108700 con 0x7f757c107d10 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.123+0000 7f757a800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:44.616 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.123+0000 7f757a800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40944/0 (socket says 172.21.15.133:40944) 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.123+0000 7f757a800700 1 -- 172.21.15.133:0/2818679510 learned_addr learned my addr 172.21.15.133:0/2818679510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.123+0000 7f757a800700 1 -- 172.21.15.133:0/2818679510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c108f10 con 0x7f757c107d10 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.124+0000 7f757a800700 1 --2- 172.21.15.133:0/2818679510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098a0 tx=0x7f756c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8773c82ce6e93a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.124+0000 7f7579400700 1 -- 172.21.15.133:0/2818679510 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f756c004160 con 0x7f757c107d10 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.124+0000 7f7579400700 1 -- 172.21.15.133:0/2818679510 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756c004300 con 0x7f757c107d10 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.124+0000 7f7579400700 1 -- 172.21.15.133:0/2818679510 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f756c011b20 con 0x7f757c107d10 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 -- 172.21.15.133:0/2818679510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 msgr2=0x7f757c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 --2- 172.21.15.133:0/2818679510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098a0 tx=0x7f756c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 -- 172.21.15.133:0/2818679510 shutdown_connections 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 --2- 172.21.15.133:0/2818679510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c108130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 -- 172.21.15.133:0/2818679510 >> 172.21.15.133:0/2818679510 conn(0x7f757c1032d0 msgr2=0x7f757c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:44.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 -- 172.21.15.133:0/2818679510 shutdown_connections 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.125+0000 7f7581400700 1 -- 172.21.15.133:0/2818679510 wait complete. 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.126+0000 7f7581400700 1 Processor -- start 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.126+0000 7f7581400700 1 -- start start 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.126+0000 7f7581400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.126+0000 7f7581400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f757c10c2c0 con 0x7f757c107d10 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f757a800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f757a800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40956/0 (socket says 172.21.15.133:40956) 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f757a800700 1 -- 172.21.15.133:0/61957323 learned_addr learned my addr 172.21.15.133:0/61957323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f757a800700 1 -- 172.21.15.133:0/61957323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757c19c0d0 con 0x7f757c107d10 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f757a800700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f756c0117e0 tx=0x7f756c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.127+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f756c003bf0 con 0x7f757c107d10 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.128+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756c01a910 con 0x7f757c107d10 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.128+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757c19c360 con 0x7f757c107d10 2024-05-09T08:19:44.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.128+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f756c0249a0 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.128+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757c19c7c0 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.129+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757c063980 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.129+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f756c022020 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.130+0000 7f7573400700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 0x7f756003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.130+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f756c04d3f0 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.130+0000 7f7579e00700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 0x7f756003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.131+0000 7f7579e00700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 0x7f756003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7568006fd0 tx=0x7f7568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.135+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f756c011e20 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.362+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f757c19f050 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.366+0000 7f7573400700 1 -- 172.21.15.133:0/61957323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f756c017cf0 con 0x7f757c107d10 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 msgr2=0x7f756003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 0x7f756003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7568006fd0 tx=0x7f7568006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 msgr2=0x7f757c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:44.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f756c0117e0 tx=0x7f756c011c10 comp rx=0 tx=0).stop 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 shutdown_connections 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f75600390c0 0x7f756003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 --2- 172.21.15.133:0/61957323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f757c107d10 0x7f757c19bb90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 >> 172.21.15.133:0/61957323 conn(0x7f757c1032d0 msgr2=0x7f757c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.369+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 shutdown_connections 2024-05-09T08:19:44.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:43.370+0000 7f7581400700 1 -- 172.21.15.133:0/61957323 wait complete. 2024-05-09T08:19:45.491 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:45 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:45.491 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:45 smithi133 ceph-mon[29610]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:19:46.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.219+0000 7f3911600700 1 Processor -- start 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.219+0000 7f3911600700 1 -- start start 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.219+0000 7f3911600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.219+0000 7f3911600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f390c07c820 con 0x7f390c07de30 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.220+0000 7f390aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.220+0000 7f390aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59038/0 (socket says 172.21.15.133:59038) 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.220+0000 7f390aa00700 1 -- 172.21.15.133:0/434185753 learned_addr learned my addr 172.21.15.133:0/434185753 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.221+0000 7f390aa00700 1 -- 172.21.15.133:0/434185753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c07c9a0 con 0x7f390c07de30 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.221+0000 7f390aa00700 1 --2- 172.21.15.133:0/434185753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0098a0 tx=0x7f38fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584826de70c078f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.221+0000 7f3909600700 1 -- 172.21.15.133:0/434185753 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc004160 con 0x7f390c07de30 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.221+0000 7f3909600700 1 -- 172.21.15.133:0/434185753 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc004300 con 0x7f390c07de30 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.221+0000 7f3909600700 1 -- 172.21.15.133:0/434185753 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc011b20 con 0x7f390c07de30 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 -- 172.21.15.133:0/434185753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 msgr2=0x7f390c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:46.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 --2- 172.21.15.133:0/434185753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0098a0 tx=0x7f38fc009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 -- 172.21.15.133:0/434185753 shutdown_connections 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 --2- 172.21.15.133:0/434185753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c07c2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 -- 172.21.15.133:0/434185753 >> 172.21.15.133:0/434185753 conn(0x7f390c1032e0 msgr2=0x7f390c105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 -- 172.21.15.133:0/434185753 shutdown_connections 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.222+0000 7f3911600700 1 -- 172.21.15.133:0/434185753 wait complete. 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.223+0000 7f3911600700 1 Processor -- start 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.223+0000 7f3911600700 1 -- start start 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f3911600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f3911600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10e300 con 0x7f390c07de30 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f390aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f390aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59054/0 (socket says 172.21.15.133:59054) 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f390aa00700 1 -- 172.21.15.133:0/4193883192 learned_addr learned my addr 172.21.15.133:0/4193883192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:46.222 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f390aa00700 1 -- 172.21.15.133:0/4193883192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c19e2e0 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.224+0000 7f390aa00700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0117e0 tx=0x7f38fc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.225+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc01a430 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.225+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38fc01aab0 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.225+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc024a60 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.225+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c19e570 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.225+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1a1220 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.226+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f38fc022020 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.226+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c063980 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.227+0000 7f3903400700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 0x7f38f003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.227+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38fc04d920 con 0x7f390c07de30 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.227+0000 7f390a000700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 0x7f38f003b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.228+0000 7f390a000700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 0x7f38f003b620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f38f8006fd0 tx=0x7f38f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:46.223 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.237+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f38fc04d270 con 0x7f390c07de30 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.459+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f390c1a13f0 con 0x7f390c07de30 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.465+0000 7f3903400700 1 -- 172.21.15.133:0/4193883192 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f38fc02ad60 con 0x7f390c07de30 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.468+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 msgr2=0x7f38f003b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.468+0000 7f3911600700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 0x7f38f003b620 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f38f8006fd0 tx=0x7f38f8006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.468+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 msgr2=0x7f390c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.468+0000 7f3911600700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0117e0 tx=0x7f38fc011ce0 comp rx=0 tx=0).stop 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 shutdown_connections 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f38f0039160 0x7f38f003b620 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 --2- 172.21.15.133:0/4193883192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f390c07de30 0x7f390c19dda0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 >> 172.21.15.133:0/4193883192 conn(0x7f390c1032e0 msgr2=0x7f390c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 shutdown_connections 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:45.469+0000 7f3911600700 1 -- 172.21.15.133:0/4193883192 wait complete. 2024-05-09T08:19:46.224 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling the dashboard module... 2024-05-09T08:19:46.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:46 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4193883192' entity='client.admin' 2024-05-09T08:19:46.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:46 smithi133 ceph-mon[29610]: from='mgr.14120 172.21.15.133:0/2718461272' entity='mgr.x' 2024-05-09T08:19:47.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:47 smithi133 ceph-mon[29610]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:19:47.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:47 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/227449038' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T08:19:47.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: ignoring --setuser ceph since I am not root 2024-05-09T08:19:47.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: ignoring --setgroup ceph since I am not root 2024-05-09T08:19:48.071 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:47.817+0000 7ff2b7aa7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:19:48.071 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:48.068+0000 7ff2b7aa7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:19:48.428 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:48.208+0000 7ff2b7aa7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:19:48.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:48 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/227449038' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T08:19:48.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:48 smithi133 ceph-mon[29610]: mgrmap e9: x(active, since 25s) 2024-05-09T08:19:48.683 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:48.623+0000 7ff2b7aa7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:19:49.097 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:48.740+0000 7ff2b7aa7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.849+0000 7ff6ea400700 1 Processor -- start 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.849+0000 7ff6ea400700 1 -- start start 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.849+0000 7ff6ea400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.850+0000 7ff6ea400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4108730 con 0x7ff6e4107d40 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.850+0000 7ff6e3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:49.113 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.850+0000 7ff6e3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59064/0 (socket says 172.21.15.133:59064) 2024-05-09T08:19:49.114 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.850+0000 7ff6e3400700 1 -- 172.21.15.133:0/3339557874 learned_addr learned my addr 172.21.15.133:0/3339557874 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:49.114 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.850+0000 7ff6e3400700 1 -- 172.21.15.133:0/3339557874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e4108ef0 con 0x7ff6e4107d40 2024-05-09T08:19:49.114 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.851+0000 7ff6e3400700 1 --2- 172.21.15.133:0/3339557874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4509e43484a42217 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:49.114 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.851+0000 7ff6e2000700 1 -- 172.21.15.133:0/3339557874 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4004160 con 0x7ff6e4107d40 2024-05-09T08:19:49.114 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.851+0000 7ff6e2000700 1 -- 172.21.15.133:0/3339557874 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d4004300 con 0x7ff6e4107d40 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.851+0000 7ff6e2000700 1 -- 172.21.15.133:0/3339557874 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4011b20 con 0x7ff6e4107d40 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 -- 172.21.15.133:0/3339557874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 msgr2=0x7ff6e4108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 --2- 172.21.15.133:0/3339557874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 -- 172.21.15.133:0/3339557874 shutdown_connections 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 --2- 172.21.15.133:0/3339557874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e4108160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:49.115 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 -- 172.21.15.133:0/3339557874 >> 172.21.15.133:0/3339557874 conn(0x7ff6e41032e0 msgr2=0x7ff6e4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 -- 172.21.15.133:0/3339557874 shutdown_connections 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.852+0000 7ff6ea400700 1 -- 172.21.15.133:0/3339557874 wait complete. 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.853+0000 7ff6ea400700 1 Processor -- start 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.853+0000 7ff6ea400700 1 -- start start 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6ea400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:49.116 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6ea400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410c2a0 con 0x7ff6e4107d40 2024-05-09T08:19:49.117 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6e3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:49.117 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6e3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59068/0 (socket says 172.21.15.133:59068) 2024-05-09T08:19:49.117 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6e3400700 1 -- 172.21.15.133:0/227449038 learned_addr learned my addr 172.21.15.133:0/227449038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:49.117 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.854+0000 7ff6e3400700 1 -- 172.21.15.133:0/227449038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e419c100 con 0x7ff6e4107d40 2024-05-09T08:19:49.117 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6e3400700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40117e0 tx=0x7ff6d4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4003bf0 con 0x7ff6e4107d40 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d401a910 con 0x7ff6e4107d40 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4024970 con 0x7ff6e4107d40 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e419c390 con 0x7ff6e4107d40 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.855+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e419c7f0 con 0x7ff6e4107d40 2024-05-09T08:19:49.118 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.856+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff6d4022020 con 0x7ff6e4107d40 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.856+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e4063980 con 0x7ff6e4107d40 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.857+0000 7ff6dbe00700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 0x7ff6c803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.857+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff6d404d3b0 con 0x7ff6e4107d40 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.857+0000 7ff6e2a00700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 0x7ff6c803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.858+0000 7ff6e2a00700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 0x7ff6c803b620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0006fd0 tx=0x7ff6d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:49.119 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:46.863+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6d401ab70 con 0x7ff6e4107d40 2024-05-09T08:19:49.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.155+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff6e419f280 con 0x7ff6e4107d40 2024-05-09T08:19:49.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.476+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff6d402d3f0 con 0x7ff6e4107d40 2024-05-09T08:19:49.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.479+0000 7ff6dbe00700 1 -- 172.21.15.133:0/227449038 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff6d401a430 con 0x7ff6e4107d40 2024-05-09T08:19:49.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 msgr2=0x7ff6c803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:49.120 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 0x7ff6c803b620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0006fd0 tx=0x7ff6d0006e40 comp rx=0 tx=0).stop 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 msgr2=0x7ff6e419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40117e0 tx=0x7ff6d4011ba0 comp rx=0 tx=0).stop 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 shutdown_connections 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7ff6c8039160 0x7ff6c803b620 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 --2- 172.21.15.133:0/227449038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6e4107d40 0x7ff6e419bbc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:49.121 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 >> 172.21.15.133:0/227449038 conn(0x7ff6e41032e0 msgr2=0x7ff6e4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:49.122 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.481+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 shutdown_connections 2024-05-09T08:19:49.122 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:47.482+0000 7ff6ea400700 1 -- 172.21.15.133:0/227449038 wait complete. 2024-05-09T08:19:50.331 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:50 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/181053312' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T08:19:50.679 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:50.328+0000 7ff2b7aa7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:19:50.680 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:50.558+0000 7ff2b7aa7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:50.695 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.696+0000 7fd66b200700 1 Processor -- start 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.696+0000 7fd66b200700 1 -- start start 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.697+0000 7fd66b200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.697+0000 7fd66b200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c108580 con 0x7fd66c107b90 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.697+0000 7fd669e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.697+0000 7fd669e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59090/0 (socket says 172.21.15.133:59090) 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.697+0000 7fd669e00700 1 -- 172.21.15.133:0/3365626674 learned_addr learned my addr 172.21.15.133:0/3365626674 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.698+0000 7fd669e00700 1 -- 172.21.15.133:0/3365626674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c108d90 con 0x7fd66c107b90 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.698+0000 7fd669e00700 1 --2- 172.21.15.133:0/3365626674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0098a0 tx=0x7fd65c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0063706eff7f34e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.698+0000 7fd668a00700 1 -- 172.21.15.133:0/3365626674 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c004160 con 0x7fd66c107b90 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.698+0000 7fd668a00700 1 -- 172.21.15.133:0/3365626674 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c004300 con 0x7fd66c107b90 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.698+0000 7fd668a00700 1 -- 172.21.15.133:0/3365626674 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c011b20 con 0x7fd66c107b90 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 -- 172.21.15.133:0/3365626674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 msgr2=0x7fd66c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 --2- 172.21.15.133:0/3365626674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0098a0 tx=0x7fd65c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 -- 172.21.15.133:0/3365626674 shutdown_connections 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 --2- 172.21.15.133:0/3365626674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c107fb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 -- 172.21.15.133:0/3365626674 >> 172.21.15.133:0/3365626674 conn(0x7fd66c103170 msgr2=0x7fd66c105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 -- 172.21.15.133:0/3365626674 shutdown_connections 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.699+0000 7fd66b200700 1 -- 172.21.15.133:0/3365626674 wait complete. 2024-05-09T08:19:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.700+0000 7fd66b200700 1 Processor -- start 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.700+0000 7fd66b200700 1 -- start start 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd66b200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd66b200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c10c140 con 0x7fd66c107b90 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd669e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd669e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59106/0 (socket says 172.21.15.133:59106) 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd669e00700 1 -- 172.21.15.133:0/181053312 learned_addr learned my addr 172.21.15.133:0/181053312 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.701+0000 7fd669e00700 1 -- 172.21.15.133:0/181053312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c07e0b0 con 0x7fd66c107b90 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd669e00700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c003b60 con 0x7fd66c107b90 2024-05-09T08:19:50.706 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c07c2a0 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd65c01a910 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd65c024930 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.702+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c07c760 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.703+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd65c022020 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.703+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c069440 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.703+0000 7fd662a00700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 0x7fd65003b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.704+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd65c04d330 con 0x7fd66c107b90 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.704+0000 7fd669400700 1 -- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 msgr2=0x7fd65003b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:50.707 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.704+0000 7fd669400700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 0x7fd65003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.713+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd65c00f2e0 con 0x7fd66c107b90 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.905+0000 7fd669400700 1 -- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 msgr2=0x7fd65003b620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.905+0000 7fd669400700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 0x7fd65003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.992+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd66c07d000 con 0x7fd66c107b90 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.992+0000 7fd662a00700 1 -- 172.21.15.133:0/181053312 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd65c02d3f0 con 0x7fd66c107b90 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 msgr2=0x7fd65003b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 0x7fd65003b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 msgr2=0x7fd66c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd65c000c00 tx=0x7fd65c011c10 comp rx=0 tx=0).stop 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 shutdown_connections 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7fd650039160 0x7fd65003b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:50.708 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 --2- 172.21.15.133:0/181053312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd66c107b90 0x7fd66c07db70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 >> 172.21.15.133:0/181053312 conn(0x7fd66c103170 msgr2=0x7fd66c103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 shutdown_connections 2024-05-09T08:19:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:49.997+0000 7fd66b200700 1 -- 172.21.15.133:0/181053312 wait complete. 2024-05-09T08:19:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2024-05-09T08:19:50.709 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 9... 2024-05-09T08:19:51.000 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:50.781+0000 7ff2b7aa7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:19:51.305 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:51.183+0000 7ff2b7aa7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:19:51.569 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:51.313+0000 7ff2b7aa7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:19:51.826 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:51.567+0000 7ff2b7aa7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:19:51.826 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:51.823+0000 7ff2b7aa7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:19:52.929 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:52 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:52.610+0000 7ff2b7aa7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:19:53.429 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:53.160+0000 7ff2b7aa7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:19:53.429 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:53.288+0000 7ff2b7aa7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:19:53.929 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:53.552+0000 7ff2b7aa7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:19:54.646 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:54.201+0000 7ff2b7aa7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:19:54.647 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:54.326+0000 7ff2b7aa7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:19:54.647 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:54.644+0000 7ff2b7aa7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:19:54.930 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:54.821+0000 7ff2b7aa7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:19:55.429 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:54.949+0000 7ff2b7aa7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:19:55.430 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:55.100+0000 7ff2b7aa7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:19:55.929 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:55.471+0000 7ff2b7aa7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:19:56.514 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:56.230+0000 7ff2b7aa7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:19:56.929 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:19:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:19:56.511+0000 7ff2b7aa7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: Active manager daemon x restarted 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: Activating manager daemon x 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: mgrmap e10: x(active, starting, since 0.0094963s) 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T08:19:56.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T08:19:56.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:56 smithi133 ceph-mon[29610]: Manager daemon x is now available 2024-05-09T08:19:57.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:19:57.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T08:19:57.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T08:19:57.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:57 smithi133 ceph-mon[29610]: mgrmap e11: x(active, since 1.00985s) 2024-05-09T08:19:58.161 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2024-05-09T08:19:58.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T08:19:58.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T08:19:58.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2024-05-09T08:19:58.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.303+0000 7f1a05600700 1 Processor -- start 2024-05-09T08:19:58.162 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.303+0000 7f1a05600700 1 -- start start 2024-05-09T08:19:58.163 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.303+0000 7f1a05600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:58.163 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.304+0000 7f1a05600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00108790 con 0x7f1a00107da0 2024-05-09T08:19:58.163 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.304+0000 7f19fea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:58.163 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.304+0000 7f19fea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59114/0 (socket says 172.21.15.133:59114) 2024-05-09T08:19:58.164 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.304+0000 7f19fea00700 1 -- 172.21.15.133:0/4125060917 learned_addr learned my addr 172.21.15.133:0/4125060917 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:58.164 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.304+0000 7f19fea00700 1 -- 172.21.15.133:0/4125060917 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a00108f00 con 0x7f1a00107da0 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.305+0000 7f19fea00700 1 --2- 172.21.15.133:0/4125060917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098a0 tx=0x7f19f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4089eefd3d63bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.305+0000 7f19fd600700 1 -- 172.21.15.133:0/4125060917 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0004160 con 0x7f1a00107da0 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.305+0000 7f19fd600700 1 -- 172.21.15.133:0/4125060917 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0004300 con 0x7f1a00107da0 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.305+0000 7f19fd600700 1 -- 172.21.15.133:0/4125060917 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0011b20 con 0x7f1a00107da0 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 -- 172.21.15.133:0/4125060917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 msgr2=0x7f1a001081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 --2- 172.21.15.133:0/4125060917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098a0 tx=0x7f19f0009bb0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 -- 172.21.15.133:0/4125060917 shutdown_connections 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 --2- 172.21.15.133:0/4125060917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a001081c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 -- 172.21.15.133:0/4125060917 >> 172.21.15.133:0/4125060917 conn(0x7f1a00103320 msgr2=0x7f1a00105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 -- 172.21.15.133:0/4125060917 shutdown_connections 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.306+0000 7f1a05600700 1 -- 172.21.15.133:0/4125060917 wait complete. 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.307+0000 7f1a05600700 1 Processor -- start 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.307+0000 7f1a05600700 1 -- start start 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f1a05600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:58.168 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f1a05600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0010c2b0 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f19fea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f19fea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59126/0 (socket says 172.21.15.133:59126) 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f19fea00700 1 -- 172.21.15.133:0/2569808545 learned_addr learned my addr 172.21.15.133:0/2569808545 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.308+0000 7f19fea00700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0007e0b0 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f19fea00700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19f00116a0 tx=0x7f19f0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f0003a40 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0003be0 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f001a720 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0007c2a0 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.309+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0007c760 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.310+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f19f0024aa0 con 0x7f1a00107da0 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.311+0000 7f19f7400700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.311+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19e403bd40 con 0x7f19e4039170 2024-05-09T08:19:58.169 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.311+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f19f004d690 con 0x7f1a00107da0 2024-05-09T08:19:58.170 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.311+0000 7f19fe000700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:58.170 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.311+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:19:58.170 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.512+0000 7f19fe000700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:58.170 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.512+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:19:58.172 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.913+0000 7f19fe000700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:58.172 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:51.913+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:19:58.172 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:52.714+0000 7f19fe000700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:58.172 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:52.714+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:19:58.172 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:54.316+0000 7f19fe000700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3643547543 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:54.316+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:56.522+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f19f0024430 con 0x7f1a00107da0 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:56.522+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 msgr2=0x7f19e403b630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:56.522+0000 7f19f7400700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.522+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f19f004f450 con 0x7f1a00107da0 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.522+0000 7f19f7400700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 0x7f19e403f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.522+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f19e403bd40 con 0x7f19e403cc30 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.523+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 0x7f19e403f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.525+0000 7f19fe000700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 0x7f19e403f020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19ec003a10 tx=0x7f19ec009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.525+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f19e403bd40 con 0x7f19e403cc30 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.534+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1a00069440 con 0x7f19e403cc30 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.534+0000 7f19f7400700 1 -- 172.21.15.133:0/2569808545 <== mgr.14152 v2:172.21.15.133:6800/2058981411 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f1a00069440 con 0x7f19e403cc30 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 msgr2=0x7f19e403f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:58.173 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 0x7f19e403f020 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f19ec003a10 tx=0x7f19ec009270 comp rx=0 tx=0).stop 2024-05-09T08:19:58.174 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 msgr2=0x7f1a0007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:19:58.174 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19f00116a0 tx=0x7f19f0011ba0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.174 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 shutdown_connections 2024-05-09T08:19:58.174 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a00107da0 0x7f1a0007db70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f19e403cc30 0x7f19e403f020 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 --2- 172.21.15.133:0/2569808545 >> [v2:172.21.15.133:6800/3643547543,v1:172.21.15.133:6801/3643547543] conn(0x7f19e4039170 0x7f19e403b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 >> 172.21.15.133:0/2569808545 conn(0x7f1a00103320 msgr2=0x7f1a00103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 shutdown_connections 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:57.535+0000 7f1a05600700 1 -- 172.21.15.133:0/2569808545 wait complete. 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 9 is available 2024-05-09T08:19:58.175 INFO:teuthology.orchestra.run.smithi133.stdout:Generating a dashboard self-signed certificate... 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: [09/May/2024:08:19:56] ENGINE Bus STARTING 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: [09/May/2024:08:19:57] ENGINE Serving on https://172.21.15.133:7150 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: [09/May/2024:08:19:57] ENGINE Serving on http://172.21.15.133:8765 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: [09/May/2024:08:19:57] ENGINE Bus STARTED 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T08:19:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:58 smithi133 ceph-mon[29610]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T08:19:59.628 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:59 smithi133 ceph-mon[29610]: mgrmap e12: x(active, since 2s) 2024-05-09T08:19:59.628 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:19:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:00.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T08:20:00.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.377+0000 7f9c71600700 1 Processor -- start 2024-05-09T08:20:00.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.377+0000 7f9c71600700 1 -- start start 2024-05-09T08:20:00.511 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.377+0000 7f9c71600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.377+0000 7f9c71600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c108700 con 0x7f9c6c107d10 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.378+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.378+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36294/0 (socket says 172.21.15.133:36294) 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.378+0000 7f9c6aa00700 1 -- 172.21.15.133:0/2708619567 learned_addr learned my addr 172.21.15.133:0/2708619567 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.378+0000 7f9c6aa00700 1 -- 172.21.15.133:0/2708619567 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c108f10 con 0x7f9c6c107d10 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.379+0000 7f9c6aa00700 1 --2- 172.21.15.133:0/2708619567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0098a0 tx=0x7f9c5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb39cd3f52c92d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.379+0000 7f9c69600700 1 -- 172.21.15.133:0/2708619567 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c004160 con 0x7f9c6c107d10 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.379+0000 7f9c69600700 1 -- 172.21.15.133:0/2708619567 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c004300 con 0x7f9c6c107d10 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.379+0000 7f9c69600700 1 -- 172.21.15.133:0/2708619567 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c011b20 con 0x7f9c6c107d10 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 -- 172.21.15.133:0/2708619567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 msgr2=0x7f9c6c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 --2- 172.21.15.133:0/2708619567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0098a0 tx=0x7f9c5c009bb0 comp rx=0 tx=0).stop 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 -- 172.21.15.133:0/2708619567 shutdown_connections 2024-05-09T08:20:00.512 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 --2- 172.21.15.133:0/2708619567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c108130 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 -- 172.21.15.133:0/2708619567 >> 172.21.15.133:0/2708619567 conn(0x7f9c6c1032d0 msgr2=0x7f9c6c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 -- 172.21.15.133:0/2708619567 shutdown_connections 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.380+0000 7f9c71600700 1 -- 172.21.15.133:0/2708619567 wait complete. 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.381+0000 7f9c71600700 1 Processor -- start 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c71600700 1 -- start start 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c71600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c71600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c10c2c0 con 0x7f9c6c107d10 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36304/0 (socket says 172.21.15.133:36304) 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.382+0000 7f9c6aa00700 1 -- 172.21.15.133:0/676774858 learned_addr learned my addr 172.21.15.133:0/676774858 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.383+0000 7f9c6aa00700 1 -- 172.21.15.133:0/676774858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c19c0d0 con 0x7f9c6c107d10 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.383+0000 7f9c6aa00700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0117e0 tx=0x7f9c5c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.383+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c003bf0 con 0x7f9c6c107d10 2024-05-09T08:20:00.513 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.384+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c01a910 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.384+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c024970 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.384+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c19c360 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.384+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c19c7c0 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.385+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f9c5c022020 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.385+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c063980 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.386+0000 7f9c63400700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 0x7f9c5003b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.386+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c5c04d570 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.386+0000 7f9c6a000700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 0x7f9c5003b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.387+0000 7f9c6a000700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 0x7f9c5003b680 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c58006fd0 tx=0x7f9c58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.391+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c5c0212c0 con 0x7f9c6c107d10 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.624+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9c6c1043b0 con 0x7f9c500391c0 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.756+0000 7f9c63400700 1 -- 172.21.15.133:0/676774858 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9c6c1043b0 con 0x7f9c500391c0 2024-05-09T08:20:00.514 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.760+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 msgr2=0x7f9c5003b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.760+0000 7f9c71600700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 0x7f9c5003b680 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c58006fd0 tx=0x7f9c58006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.760+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 msgr2=0x7f9c6c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.760+0000 7f9c71600700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0117e0 tx=0x7f9c5c011ba0 comp rx=0 tx=0).stop 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 shutdown_connections 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9c500391c0 0x7f9c5003b680 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 --2- 172.21.15.133:0/676774858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c6c107d10 0x7f9c6c19bb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 >> 172.21.15.133:0/676774858 conn(0x7f9c6c1032d0 msgr2=0x7f9c6c103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 shutdown_connections 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:19:59.761+0000 7f9c71600700 1 -- 172.21.15.133:0/676774858 wait complete. 2024-05-09T08:20:00.515 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial admin user... 2024-05-09T08:20:00.763 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:00 smithi133 ceph-mon[29610]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:00.763 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:00 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:00.763 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:00 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:02.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:02 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:02.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:02 smithi133 ceph-mon[29610]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:02.411 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:02 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:02.420 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$nbsF3/HmIHe.BV/Aizke4.kSvGExT5IKxX3WbK9JfQVLmXCk5TGbS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715242801, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T08:20:02.420 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.197+0000 7f6f79a00700 1 Processor -- start 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.198+0000 7f6f79a00700 1 -- start start 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.198+0000 7f6f79a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.198+0000 7f6f79a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f741085d0 con 0x7f6f74107be0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.198+0000 7f6f73e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.199+0000 7f6f73e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36310/0 (socket says 172.21.15.133:36310) 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.199+0000 7f6f73e00700 1 -- 172.21.15.133:0/3317865530 learned_addr learned my addr 172.21.15.133:0/3317865530 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.199+0000 7f6f73e00700 1 -- 172.21.15.133:0/3317865530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f74108de0 con 0x7f6f74107be0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.199+0000 7f6f73e00700 1 --2- 172.21.15.133:0/3317865530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098a0 tx=0x7f6f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963cd30b0d9607d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f72a00700 1 -- 172.21.15.133:0/3317865530 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f64004160 con 0x7f6f74107be0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f72a00700 1 -- 172.21.15.133:0/3317865530 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f64004300 con 0x7f6f74107be0 2024-05-09T08:20:02.426 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f72a00700 1 -- 172.21.15.133:0/3317865530 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f64011b20 con 0x7f6f74107be0 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f79a00700 1 -- 172.21.15.133:0/3317865530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 msgr2=0x7f6f74108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3317865530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f640098a0 tx=0x7f6f64009bb0 comp rx=0 tx=0).stop 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f79a00700 1 -- 172.21.15.133:0/3317865530 shutdown_connections 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3317865530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f74108000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.200+0000 7f6f79a00700 1 -- 172.21.15.133:0/3317865530 >> 172.21.15.133:0/3317865530 conn(0x7f6f74103220 msgr2=0x7f6f74105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.201+0000 7f6f79a00700 1 -- 172.21.15.133:0/3317865530 shutdown_connections 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.201+0000 7f6f79a00700 1 -- 172.21.15.133:0/3317865530 wait complete. 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.202+0000 7f6f79a00700 1 Processor -- start 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.202+0000 7f6f79a00700 1 -- start start 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.202+0000 7f6f79a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.202+0000 7f6f79a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410c190 con 0x7f6f74107be0 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f73e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f73e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36320/0 (socket says 172.21.15.133:36320) 2024-05-09T08:20:02.427 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f73e00700 1 -- 172.21.15.133:0/3234555820 learned_addr learned my addr 172.21.15.133:0/3234555820 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f73e00700 1 -- 172.21.15.133:0/3234555820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7419bf90 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f73e00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000c00 tx=0x7f6f64011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f64003b60 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6401a910 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f64024930 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.203+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7419c220 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.204+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7419c680 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.205+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6f64022020 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.205+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f74069440 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.206+0000 7f6f70c00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 0x7f6f5803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.206+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f6404d390 con 0x7f6f74107be0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.206+0000 7f6f73400700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 0x7f6f5803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.207+0000 7f6f73400700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 0x7f6f5803b680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006fd0 tx=0x7f6f60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:02.428 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.212+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f6400f2e0 con 0x7f6f74107be0 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.453+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6f7419efd0 con 0x7f6f580391c0 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.710+0000 7f6f70c00700 1 -- 172.21.15.133:0/3234555820 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6f7419efd0 con 0x7f6f580391c0 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.714+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 msgr2=0x7f6f5803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 0x7f6f5803b680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006fd0 tx=0x7f6f60006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 msgr2=0x7f6f7419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000c00 tx=0x7f6f64011c10 comp rx=0 tx=0).stop 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 shutdown_connections 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f580391c0 0x7f6f5803b680 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 --2- 172.21.15.133:0/3234555820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f74107be0 0x7f6f7419ba50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 >> 172.21.15.133:0/3234555820 conn(0x7f6f74103220 msgr2=0x7f6f74103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 shutdown_connections 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:01.715+0000 7f6f79a00700 1 -- 172.21.15.133:0/3234555820 wait complete. 2024-05-09T08:20:02.429 INFO:teuthology.orchestra.run.smithi133.stdout:Fetching dashboard port number... 2024-05-09T08:20:04.611 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:04 smithi133 ceph-mon[29610]: mgrmap e13: x(active, since 6s) 2024-05-09T08:20:04.612 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:04 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4038859310' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T08:20:04.679 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d15600700 1 Processor -- start 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d15600700 1 -- start start 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d15600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d15600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1007c890 con 0x7f4d1007de60 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d0ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d0ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36326/0 (socket says 172.21.15.133:36326) 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.117+0000 7f4d0ea00700 1 -- 172.21.15.133:0/4239121164 learned_addr learned my addr 172.21.15.133:0/4239121164 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.118+0000 7f4d0ea00700 1 -- 172.21.15.133:0/4239121164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1007ca10 con 0x7f4d1007de60 2024-05-09T08:20:04.685 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.118+0000 7f4d0ea00700 1 --2- 172.21.15.133:0/4239121164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d000098a0 tx=0x7f4d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd12d6f2cb734a60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.118+0000 7f4d0d600700 1 -- 172.21.15.133:0/4239121164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00004160 con 0x7f4d1007de60 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d0d600700 1 -- 172.21.15.133:0/4239121164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d00004300 con 0x7f4d1007de60 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d0d600700 1 -- 172.21.15.133:0/4239121164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00011b20 con 0x7f4d1007de60 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 -- 172.21.15.133:0/4239121164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 msgr2=0x7f4d1007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 --2- 172.21.15.133:0/4239121164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d000098a0 tx=0x7f4d00009bb0 comp rx=0 tx=0).stop 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 -- 172.21.15.133:0/4239121164 shutdown_connections 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 --2- 172.21.15.133:0/4239121164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1007c2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 -- 172.21.15.133:0/4239121164 >> 172.21.15.133:0/4239121164 conn(0x7f4d101032d0 msgr2=0x7f4d101056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.119+0000 7f4d15600700 1 -- 172.21.15.133:0/4239121164 shutdown_connections 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.120+0000 7f4d15600700 1 -- 172.21.15.133:0/4239121164 wait complete. 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.120+0000 7f4d15600700 1 Processor -- start 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d15600700 1 -- start start 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d15600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:04.686 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d15600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1010e350 con 0x7f4d1007de60 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d0ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d0ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36328/0 (socket says 172.21.15.133:36328) 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d0ea00700 1 -- 172.21.15.133:0/4038859310 learned_addr learned my addr 172.21.15.133:0/4038859310 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.121+0000 7f4d0ea00700 1 -- 172.21.15.133:0/4038859310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1019e2f0 con 0x7f4d1007de60 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d0ea00700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d000117e0 tx=0x7f4d00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00003bf0 con 0x7f4d1007de60 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0001a910 con 0x7f4d1007de60 2024-05-09T08:20:04.687 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00024970 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1019e580 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.122+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d101a1230 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.123+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4d00022020 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.123+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d10063980 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.124+0000 7f4d07400700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 0x7f4cf403b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.124+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d0004d490 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.124+0000 7f4d0e000700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 0x7f4cf403b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.125+0000 7f4d0e000700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 0x7f4cf403b630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006fd0 tx=0x7f4cfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.129+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d00011e20 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.223+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4d0001a430 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.357+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4d1019e9e0 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.358+0000 7f4d07400700 1 -- 172.21.15.133:0/4038859310 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4d0000b3e0 con 0x7f4d1007de60 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.361+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 msgr2=0x7f4cf403b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:04.688 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.361+0000 7f4d15600700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 0x7f4cf403b630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006fd0 tx=0x7f4cfc006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:04.689 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.361+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 msgr2=0x7f4d1019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:04.689 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.361+0000 7f4d15600700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d000117e0 tx=0x7f4d00011c10 comp rx=0 tx=0).stop 2024-05-09T08:20:04.689 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 shutdown_connections 2024-05-09T08:20:04.690 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4cf4039170 0x7f4cf403b630 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:04.690 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 --2- 172.21.15.133:0/4038859310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1007de60 0x7f4d1019ddb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:04.690 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 >> 172.21.15.133:0/4038859310 conn(0x7f4d101032d0 msgr2=0x7f4d10103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:04.690 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 shutdown_connections 2024-05-09T08:20:04.690 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:03.362+0000 7f4d15600700 1 -- 172.21.15.133:0/4038859310 wait complete. 2024-05-09T08:20:04.711 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T08:20:04.711 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2024-05-09T08:20:04.736 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T08:20:04.737 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2024-05-09T08:20:04.737 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2024-05-09T08:20:04.737 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T08:20:04.744 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph Dashboard is now available at: 2024-05-09T08:20:04.744 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:04.744 INFO:teuthology.orchestra.run.smithi133.stdout: URL: https://smithi133.front.sepia.ceph.com:8443/ 2024-05-09T08:20:04.744 INFO:teuthology.orchestra.run.smithi133.stdout: User: admin 2024-05-09T08:20:04.744 INFO:teuthology.orchestra.run.smithi133.stdout: Password: 6z9wxpvuv7 2024-05-09T08:20:04.745 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:04.745 INFO:teuthology.orchestra.run.smithi133.stdout:Saving cluster configuration to /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config directory 2024-05-09T08:20:04.747 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling autotune for osd_memory_target 2024-05-09T08:20:06.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.416+0000 7f53ace00700 1 Processor -- start 2024-05-09T08:20:06.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53ace00700 1 -- start start 2024-05-09T08:20:06.344 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53ace00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53ace00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8107610 con 0x7f53a8106c20 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48250/0 (socket says 172.21.15.133:48250) 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.417+0000 7f53a5e00700 1 -- 172.21.15.133:0/167016134 learned_addr learned my addr 172.21.15.133:0/167016134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.418+0000 7f53a5e00700 1 -- 172.21.15.133:0/167016134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a8107e20 con 0x7f53a8106c20 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.418+0000 7f53a5e00700 1 --2- 172.21.15.133:0/167016134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53980098a0 tx=0x7f5398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a6da30ade3e71e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.419+0000 7f53a4a00700 1 -- 172.21.15.133:0/167016134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398004160 con 0x7f53a8106c20 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.419+0000 7f53a4a00700 1 -- 172.21.15.133:0/167016134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5398004300 con 0x7f53a8106c20 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.419+0000 7f53a4a00700 1 -- 172.21.15.133:0/167016134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398011ad0 con 0x7f53a8106c20 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.419+0000 7f53ace00700 1 -- 172.21.15.133:0/167016134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 msgr2=0x7f53a8107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.419+0000 7f53ace00700 1 --2- 172.21.15.133:0/167016134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53980098a0 tx=0x7f5398009bb0 comp rx=0 tx=0).stop 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.420+0000 7f53ace00700 1 -- 172.21.15.133:0/167016134 shutdown_connections 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.420+0000 7f53ace00700 1 --2- 172.21.15.133:0/167016134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8107040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:06.345 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.420+0000 7f53ace00700 1 -- 172.21.15.133:0/167016134 >> 172.21.15.133:0/167016134 conn(0x7f53a81021e0 msgr2=0x7f53a8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.420+0000 7f53ace00700 1 -- 172.21.15.133:0/167016134 shutdown_connections 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.420+0000 7f53ace00700 1 -- 172.21.15.133:0/167016134 wait complete. 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.421+0000 7f53ace00700 1 Processor -- start 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.421+0000 7f53ace00700 1 -- start start 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.421+0000 7f53ace00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.421+0000 7f53ace00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53a810b1d0 con 0x7f53a8106c20 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.421+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f53a5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48260/0 (socket says 172.21.15.133:48260) 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f53a5e00700 1 -- 172.21.15.133:0/4123103452 learned_addr learned my addr 172.21.15.133:0/4123103452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f53a5e00700 1 -- 172.21.15.133:0/4123103452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a8192310 con 0x7f53a8106c20 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f53a5e00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5398000c00 tx=0x7f5398004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398003ad0 con 0x7f53a8106c20 2024-05-09T08:20:06.346 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.422+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5398003c70 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.423+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5398019750 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.423+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a8190500 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.423+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81909c0 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.424+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a8063980 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.424+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f539801ab30 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.424+0000 7f539ea00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 0x7f538c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.425+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f539804d9f0 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.425+0000 7f53a5400700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 0x7f538c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.425+0000 7f53a5400700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 0x7f538c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.430+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f539800b3e0 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.652+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f53a8195b90 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.653+0000 7f539ea00700 1 -- 172.21.15.133:0/4123103452 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f5398004300 con 0x7f53a8106c20 2024-05-09T08:20:06.347 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 msgr2=0x7f538c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 0x7f538c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 msgr2=0x7f53a8191dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5398000c00 tx=0x7f5398004280 comp rx=0 tx=0).stop 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 shutdown_connections 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f538c039200 0x7f538c03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 --2- 172.21.15.133:0/4123103452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8106c20 0x7f53a8191dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.656+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 >> 172.21.15.133:0/4123103452 conn(0x7f53a81021e0 msgr2=0x7f53a8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.657+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 shutdown_connections 2024-05-09T08:20:06.348 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:05.657+0000 7f53ace00700 1 -- 172.21.15.133:0/4123103452 wait complete. 2024-05-09T08:20:08.102 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.021+0000 7f14fe600700 1 Processor -- start 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.021+0000 7f14fe600700 1 -- start start 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.022+0000 7f14fe600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.022+0000 7f14fe600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f14f807c8f0 con 0x7f14f807dec0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.022+0000 7f14f7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.022+0000 7f14f7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48264/0 (socket says 172.21.15.133:48264) 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.022+0000 7f14f7400700 1 -- 172.21.15.133:0/4082296436 learned_addr learned my addr 172.21.15.133:0/4082296436 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.023+0000 7f14f7400700 1 -- 172.21.15.133:0/4082296436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f807ca70 con 0x7f14f807dec0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.023+0000 7f14f7400700 1 --2- 172.21.15.133:0/4082296436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14e80098a0 tx=0x7f14e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa766e27e0a5406 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.023+0000 7f14f6000700 1 -- 172.21.15.133:0/4082296436 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8004160 con 0x7f14f807dec0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.023+0000 7f14f6000700 1 -- 172.21.15.133:0/4082296436 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8004300 con 0x7f14f807dec0 2024-05-09T08:20:08.103 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.023+0000 7f14f6000700 1 -- 172.21.15.133:0/4082296436 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8011b20 con 0x7f14f807dec0 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 -- 172.21.15.133:0/4082296436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 msgr2=0x7f14f807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 --2- 172.21.15.133:0/4082296436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14e80098a0 tx=0x7f14e8009bb0 comp rx=0 tx=0).stop 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 -- 172.21.15.133:0/4082296436 shutdown_connections 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 --2- 172.21.15.133:0/4082296436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f807c320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 -- 172.21.15.133:0/4082296436 >> 172.21.15.133:0/4082296436 conn(0x7f14f81032d0 msgr2=0x7f14f81056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 -- 172.21.15.133:0/4082296436 shutdown_connections 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.024+0000 7f14fe600700 1 -- 172.21.15.133:0/4082296436 wait complete. 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.025+0000 7f14fe600700 1 Processor -- start 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.025+0000 7f14fe600700 1 -- start start 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14fe600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14fe600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f14f810e350 con 0x7f14f807dec0 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14f7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14f7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48266/0 (socket says 172.21.15.133:48266) 2024-05-09T08:20:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14f7400700 1 -- 172.21.15.133:0/3810867076 learned_addr learned my addr 172.21.15.133:0/3810867076 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.026+0000 7f14f7400700 1 -- 172.21.15.133:0/3810867076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f819e2f0 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14f7400700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f14e80117e0 tx=0x7f14e8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e8003bf0 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e801a910 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e80249a0 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f819e580 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.027+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81a1230 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.028+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f14e8022020 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.028+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f8063980 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.029+0000 7f14efe00700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 0x7f14dc03b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.029+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14e804d540 con 0x7f14f807dec0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.029+0000 7f14f6a00700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 0x7f14dc03b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:08.105 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.030+0000 7f14f6a00700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 0x7f14dc03b6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.034+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14e8011e20 con 0x7f14f807dec0 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.374+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f14f81a1400 con 0x7f14f807dec0 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.379+0000 7f14efe00700 1 -- 172.21.15.133:0/3810867076 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f14e8004300 con 0x7f14f807dec0 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 msgr2=0x7f14dc03b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 0x7f14dc03b6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14e4006fd0 tx=0x7f14e4006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 msgr2=0x7f14f819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f14e80117e0 tx=0x7f14e8011c10 comp rx=0 tx=0).stop 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 shutdown_connections 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f14dc039210 0x7f14dc03b6d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 --2- 172.21.15.133:0/3810867076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14f807dec0 0x7f14f819ddb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 >> 172.21.15.133:0/3810867076 conn(0x7f14f81032d0 msgr2=0x7f14f8103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 shutdown_connections 2024-05-09T08:20:08.106 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2024-05-09T08:20:07.383+0000 7f14fe600700 1 -- 172.21.15.133:0/3810867076 wait complete. 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: ceph telemetry on 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout:For more information see: 2024-05-09T08:20:08.107 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.108 INFO:teuthology.orchestra.run.smithi133.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T08:20:08.108 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:08.108 INFO:teuthology.orchestra.run.smithi133.stdout:Bootstrap complete. 2024-05-09T08:20:08.177 INFO:tasks.cephadm:Fetching config... 2024-05-09T08:20:08.177 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:20:08.178 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T08:20:08.227 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T08:20:08.227 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:20:08.228 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T08:20:08.307 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T08:20:08.307 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:20:08.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T08:20:08.425 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T08:20:08.426 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:20:08.426 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T08:20:08.517 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T08:20:08.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:20:08.619 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:08.648 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:08 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3810867076' entity='client.admin' 2024-05-09T08:20:08.665 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:20:08.698 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:08.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T08:20:08.756 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCql2mGIjQP2/4Gv3k1LloXboF2hkvr/vh3dYmrSrrGfzzuxzAUtlI/MUzCV/H3riebenzffDcKA1XFNnB2scPH7KgZ3XHaM9lsCvk2YFnZD5pvX+6HrA9ltGU8sQcuaZcCD0WgqASrfgNOwPSZ/togSynvGVXIE7eq6WTFSRrrtM70rJJ9avrHj2SDXktk4cfVuolpBmpaHOYNo6LMehLSwV3eYLgEuBXtAKZwEUVsf8GTyqGgZr3tfSzLmmykSkJVNiavUXHZUbZTJ9yE5dV33H3nvf+Ntuapn5bdqVnZOfZoFTPxLH5gkupyrW84nnMo1OUeptyLvOhflBvyQsHaP1vF8Dnn+gJ+BG4mEcTxS52MQk16e1wzqlIgy/xL9KAyUEph/6BrS+Ly3NH6peywRRKBLCo/Za36cpCnZK1ACyrIzfkQeuJvo1GPLCgA9xqE8tYE2JMz9CTa6PQ1DGyQwvWQGC98LoDfTitPnc3tCIBj1R3+4KjSq65ubF2YO5E= ceph-99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:08.769 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T08:20:09.172 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:12.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 -- 172.21.15.133:0/4233198089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 msgr2=0x7f00140747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:12.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 --2- 172.21.15.133:0/4233198089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f00140747b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0004009910 tx=0x7f0004009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:12.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 -- 172.21.15.133:0/4233198089 shutdown_connections 2024-05-09T08:20:12.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 --2- 172.21.15.133:0/4233198089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f00140747b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:12.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 -- 172.21.15.133:0/4233198089 >> 172.21.15.133:0/4233198089 conn(0x7f00140fd310 msgr2=0x7f00140ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:12.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 -- 172.21.15.133:0/4233198089 shutdown_connections 2024-05-09T08:20:12.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.035+0000 7f001a000700 1 -- 172.21.15.133:0/4233198089 wait complete. 2024-05-09T08:20:12.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.036+0000 7f001a000700 1 Processor -- start 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.036+0000 7f001a000700 1 -- start start 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f001a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f001a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f001410e4d0 con 0x7f0014076350 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f0013400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f0013400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48282/0 (socket says 172.21.15.133:48282) 2024-05-09T08:20:12.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f0013400700 1 -- 172.21.15.133:0/1856625873 learned_addr learned my addr 172.21.15.133:0/1856625873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f0013400700 1 -- 172.21.15.133:0/1856625873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001419dfb0 con 0x7f0014076350 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.037+0000 7f0013400700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0004006010 tx=0x7f0004005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.038+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0004027070 con 0x7f0014076350 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.038+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000400be40 con 0x7f0014076350 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.038+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f000400f460 con 0x7f0014076350 2024-05-09T08:20:12.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.038+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001419e240 con 0x7f0014076350 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.038+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141a0f30 con 0x7f0014076350 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.039+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f000402a070 con 0x7f0014076350 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.039+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001410eb80 con 0x7f0014076350 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.040+0000 7f000be00700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 0x7efff803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.040+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f000401e030 con 0x7f0014076350 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.040+0000 7f0012a00700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 0x7efff803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:12.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.041+0000 7f0012a00700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 0x7efff803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0000006fd0 tx=0x7f0000006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:12.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.045+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0004013070 con 0x7f0014076350 2024-05-09T08:20:12.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.268+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f001402dc90 con 0x7f0014076350 2024-05-09T08:20:12.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.274+0000 7f000be00700 1 -- 172.21.15.133:0/1856625873 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0004031020 con 0x7f0014076350 2024-05-09T08:20:12.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 msgr2=0x7efff803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:12.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 0x7efff803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0000006fd0 tx=0x7f0000006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:12.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 msgr2=0x7f001419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:12.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0004006010 tx=0x7f0004005050 comp rx=0 tx=0).stop 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 shutdown_connections 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efff80390f0 0x7efff803b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 --2- 172.21.15.133:0/1856625873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0014076350 0x7f001419da70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.277+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 >> 172.21.15.133:0/1856625873 conn(0x7f00140fd310 msgr2=0x7f00140fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.278+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 shutdown_connections 2024-05-09T08:20:12.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:12.278+0000 7f001a000700 1 -- 172.21.15.133:0/1856625873 wait complete. 2024-05-09T08:20:12.672 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:12.672 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:12.672 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1856625873' entity='client.admin' 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:12.673 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:13.081 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T08:20:13.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T08:20:13.686 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:16.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 -- 172.21.15.133:0/893089765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 msgr2=0x7f092c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 --2- 172.21.15.133:0/893089765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c102a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f091c009910 tx=0x7f091c009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 -- 172.21.15.133:0/893089765 shutdown_connections 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 --2- 172.21.15.133:0/893089765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c102a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 -- 172.21.15.133:0/893089765 >> 172.21.15.133:0/893089765 conn(0x7f092c0fc230 msgr2=0x7f092c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 -- 172.21.15.133:0/893089765 shutdown_connections 2024-05-09T08:20:16.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.047+0000 7f0930a00700 1 -- 172.21.15.133:0/893089765 wait complete. 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.048+0000 7f0930a00700 1 Processor -- start 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0930a00700 1 -- start start 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0930a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0930a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f092c105cd0 con 0x7f092c100610 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0929e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0929e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40716/0 (socket says 172.21.15.133:40716) 2024-05-09T08:20:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.049+0000 7f0929e00700 1 -- 172.21.15.133:0/259203144 learned_addr learned my addr 172.21.15.133:0/259203144 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:16.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0929e00700 1 -- 172.21.15.133:0/259203144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c19dff0 con 0x7f092c100610 2024-05-09T08:20:16.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0929e00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f091c005950 tx=0x7f091c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:16.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f091c026070 con 0x7f092c100610 2024-05-09T08:20:16.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f091c00bc30 con 0x7f092c100610 2024-05-09T08:20:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c19e280 con 0x7f092c100610 2024-05-09T08:20:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.050+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f091c003b30 con 0x7f092c100610 2024-05-09T08:20:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.051+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c1a0f70 con 0x7f092c100610 2024-05-09T08:20:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.052+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f091c029070 con 0x7f092c100610 2024-05-09T08:20:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.052+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c106450 con 0x7f092c100610 2024-05-09T08:20:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.052+0000 7f0922a00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 0x7f091003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.052+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f091c01e070 con 0x7f092c100610 2024-05-09T08:20:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.052+0000 7f0929400700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 0x7f091003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.053+0000 7f0929400700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 0x7f091003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0918006fd0 tx=0x7f0918006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:16.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.058+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f091c013070 con 0x7f092c100610 2024-05-09T08:20:16.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.290+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f092c02dd20 con 0x7f0910039150 2024-05-09T08:20:16.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.296+0000 7f0922a00700 1 -- 172.21.15.133:0/259203144 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f092c02dd20 con 0x7f0910039150 2024-05-09T08:20:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 msgr2=0x7f091003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 0x7f091003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0918006fd0 tx=0x7f0918006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 msgr2=0x7f092c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f091c005950 tx=0x7f091c005030 comp rx=0 tx=0).stop 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 shutdown_connections 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0910039150 0x7f091003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 --2- 172.21.15.133:0/259203144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c100610 0x7f092c19dab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 >> 172.21.15.133:0/259203144 conn(0x7f092c0fc230 msgr2=0x7f092c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.299+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 shutdown_connections 2024-05-09T08:20:16.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:16.300+0000 7f0930a00700 1 -- 172.21.15.133:0/259203144 wait complete. 2024-05-09T08:20:17.090 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2024-05-09T08:20:17.091 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:20:17.091 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2024-05-09T08:20:17.112 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:20:17.112 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:17.179 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2024-05-09T08:20:17.179 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch host add smithi160 2024-05-09T08:20:17.578 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:17.690 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:17.690 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:17.691 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:17.691 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:17.691 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:17.691 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: Updating smithi133:/etc/ceph/ceph.conf 2024-05-09T08:20:17.691 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:17 smithi133 ceph-mon[29610]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:18.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:18 smithi133 ceph-mon[29610]: Updating smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:18.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:18 smithi133 ceph-mon[29610]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 -- 172.21.15.133:0/1781906982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 msgr2=0x7fbbc010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1781906982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc010e7f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009910 tx=0x7fbbb0009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 -- 172.21.15.133:0/1781906982 shutdown_connections 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1781906982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc010e7f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 -- 172.21.15.133:0/1781906982 >> 172.21.15.133:0/1781906982 conn(0x7fbbc0077b80 msgr2=0x7fbbc0079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 -- 172.21.15.133:0/1781906982 shutdown_connections 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.702+0000 7fbbc7200700 1 -- 172.21.15.133:0/1781906982 wait complete. 2024-05-09T08:20:19.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.703+0000 7fbbc7200700 1 Processor -- start 2024-05-09T08:20:19.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.703+0000 7fbbc7200700 1 -- start start 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbc7200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbc7200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0112960 con 0x7fbbc010e410 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbbfe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbbfe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40722/0 (socket says 172.21.15.133:40722) 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbbfe00700 1 -- 172.21.15.133:0/1454307642 learned_addr learned my addr 172.21.15.133:0/1454307642 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.704+0000 7fbbbfe00700 1 -- 172.21.15.133:0/1454307642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc019e300 con 0x7fbbc010e410 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbbfe00700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:19.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbb0026070 con 0x7fbbc010e410 2024-05-09T08:20:19.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb000bbd0 con 0x7fbbc010e410 2024-05-09T08:20:19.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc019e5f0 con 0x7fbbc010e410 2024-05-09T08:20:19.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbb00039d0 con 0x7fbbc010e410 2024-05-09T08:20:19.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.705+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01a2690 con 0x7fbbc010e410 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.706+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbbb0029070 con 0x7fbbc010e410 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.706+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc0109a50 con 0x7fbbc010e410 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.707+0000 7fbbbcc00700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 0x7fbba403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.707+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbbb001e070 con 0x7fbbc010e410 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.707+0000 7fbbbf400700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 0x7fbba403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:19.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.708+0000 7fbbbf400700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 0x7fbba403b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbac006fd0 tx=0x7fbbac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:19.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.712+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbbb0013070 con 0x7fbbc010e410 2024-05-09T08:20:19.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:19 smithi133 ceph-mon[29610]: Updating smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:20:19.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:19 smithi133 ceph-mon[29610]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:19.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:19.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:19.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:19.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:19.942+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) -- 0x7fbbc019f190 con 0x7fbba4039150 2024-05-09T08:20:21.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:21 smithi133 ceph-mon[29610]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:21.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:21 smithi133 ceph-mon[29610]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:22.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:22 smithi133 ceph-mon[29610]: Deploying cephadm binary to smithi160 2024-05-09T08:20:23.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:23 smithi133 ceph-mon[29610]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:24.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.610+0000 7fbbbcc00700 1 -- 172.21.15.133:0/1454307642 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbbc019f190 con 0x7fbba4039150 2024-05-09T08:20:24.613 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi160' with addr '172.21.15.160' 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 msgr2=0x7fbba403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 0x7fbba403b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbac006fd0 tx=0x7fbbac006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 msgr2=0x7fbbc019ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0006010 tx=0x7fbbb0005030 comp rx=0 tx=0).stop 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 shutdown_connections 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbba4039150 0x7fbba403b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 --2- 172.21.15.133:0/1454307642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbc010e410 0x7fbbc019ddc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.614+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 >> 172.21.15.133:0/1454307642 conn(0x7fbbc0077b80 msgr2=0x7fbbc00793d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.615+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 shutdown_connections 2024-05-09T08:20:24.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:24.615+0000 7fbbc7200700 1 -- 172.21.15.133:0/1454307642 wait complete. 2024-05-09T08:20:25.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T08:20:25.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:25 smithi133 ceph-mon[29610]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:25.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:25.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:25 smithi133 ceph-mon[29610]: Added host smithi160 2024-05-09T08:20:25.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:25.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:25.909 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:27.122 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:26 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:27.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:26 smithi133 ceph-mon[29610]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.916+0000 7fc6f8a00700 1 -- 172.21.15.133:0/2602637094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 msgr2=0x7fc6f4104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.916+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/2602637094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f4104560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009910 tx=0x7fc6e4009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.917+0000 7fc6f8a00700 1 -- 172.21.15.133:0/2602637094 shutdown_connections 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.917+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/2602637094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f4104560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.917+0000 7fc6f8a00700 1 -- 172.21.15.133:0/2602637094 >> 172.21.15.133:0/2602637094 conn(0x7fc6f40ffa30 msgr2=0x7fc6f4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.917+0000 7fc6f8a00700 1 -- 172.21.15.133:0/2602637094 shutdown_connections 2024-05-09T08:20:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.917+0000 7fc6f8a00700 1 -- 172.21.15.133:0/2602637094 wait complete. 2024-05-09T08:20:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.918+0000 7fc6f8a00700 1 Processor -- start 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.918+0000 7fc6f8a00700 1 -- start start 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.918+0000 7fc6f8a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.918+0000 7fc6f8a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f410e550 con 0x7fc6f4104180 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.919+0000 7fc6f1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.919+0000 7fc6f1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44760/0 (socket says 172.21.15.133:44760) 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.919+0000 7fc6f1e00700 1 -- 172.21.15.133:0/1935636786 learned_addr learned my addr 172.21.15.133:0/1935636786 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:27.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.919+0000 7fc6f1e00700 1 -- 172.21.15.133:0/1935636786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f419fc10 con 0x7fc6f4104180 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.919+0000 7fc6f1e00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005850 tx=0x7fc6e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.920+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6e4026070 con 0x7fc6f4104180 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.920+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e400bbd0 con 0x7fc6f4104180 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.920+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6e4003970 con 0x7fc6f4104180 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.920+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f419ff00 con 0x7fc6f4104180 2024-05-09T08:20:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.920+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f4199890 con 0x7fc6f4104180 2024-05-09T08:20:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.921+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc6e4029070 con 0x7fc6f4104180 2024-05-09T08:20:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.921+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4105660 con 0x7fc6f4104180 2024-05-09T08:20:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.922+0000 7fc6eaa00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 0x7fc6d803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.922+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6e401e070 con 0x7fc6f4104180 2024-05-09T08:20:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.922+0000 7fc6f1400700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 0x7fc6d803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.922+0000 7fc6f1400700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 0x7fc6d803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0006fd0 tx=0x7fc6e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:27.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:27.927+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6e4013070 con 0x7fc6f4104180 2024-05-09T08:20:28.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.158+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6f419adc0 con 0x7fc6d8039150 2024-05-09T08:20:28.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.161+0000 7fc6eaa00700 1 -- 172.21.15.133:0/1935636786 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fc6f419adc0 con 0x7fc6d8039150 2024-05-09T08:20:28.163 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:28.163 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2024-05-09T08:20:28.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 msgr2=0x7fc6d803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 0x7fc6d803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0006fd0 tx=0x7fc6e0006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 msgr2=0x7fc6f419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005850 tx=0x7fc6e4005030 comp rx=0 tx=0).stop 2024-05-09T08:20:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 shutdown_connections 2024-05-09T08:20:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6d8039150 0x7fc6d803b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:28.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 --2- 172.21.15.133:0/1935636786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6f4104180 0x7fc6f419f6d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:28.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 >> 172.21.15.133:0/1935636786 conn(0x7fc6f40ffa30 msgr2=0x7fc6f4105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:28.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 shutdown_connections 2024-05-09T08:20:28.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:28.164+0000 7fc6f8a00700 1 -- 172.21.15.133:0/1935636786 wait complete. 2024-05-09T08:20:28.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:28.880 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-05-09T08:20:28.880 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:20:28.881 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-05-09T08:20:28.899 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:20:28.900 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:28.957 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-05-09T08:20:28.957 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch host add smithi188 2024-05-09T08:20:29.391 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:29.554 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:29 smithi133 ceph-mon[29610]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:20:29.555 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:29 smithi133 ceph-mon[29610]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.395+0000 7f9af1e00700 1 -- 172.21.15.133:0/3940401744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 msgr2=0x7f9aec10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.395+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3940401744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec10e7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009910 tx=0x7f9adc009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.395+0000 7f9af1e00700 1 -- 172.21.15.133:0/3940401744 shutdown_connections 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.395+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3940401744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec10e7f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.395+0000 7f9af1e00700 1 -- 172.21.15.133:0/3940401744 >> 172.21.15.133:0/3940401744 conn(0x7f9aec077b80 msgr2=0x7f9aec079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.396+0000 7f9af1e00700 1 -- 172.21.15.133:0/3940401744 shutdown_connections 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.396+0000 7f9af1e00700 1 -- 172.21.15.133:0/3940401744 wait complete. 2024-05-09T08:20:31.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.397+0000 7f9af1e00700 1 Processor -- start 2024-05-09T08:20:31.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.397+0000 7f9af1e00700 1 -- start start 2024-05-09T08:20:31.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.397+0000 7f9af1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:31.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.397+0000 7f9af1e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec112960 con 0x7f9aec10e410 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.398+0000 7f9aeaa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.398+0000 7f9aeaa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44784/0 (socket says 172.21.15.133:44784) 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.398+0000 7f9aeaa00700 1 -- 172.21.15.133:0/3684229359 learned_addr learned my addr 172.21.15.133:0/3684229359 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.398+0000 7f9aeaa00700 1 -- 172.21.15.133:0/3684229359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec19e300 con 0x7f9aec10e410 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.398+0000 7f9aeaa00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006010 tx=0x7f9adc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:31.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.399+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9adc026070 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.399+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc00bbd0 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.399+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec19e5f0 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.399+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9adc0039d0 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.399+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1a2690 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.400+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec109a50 con 0x7f9aec10e410 2024-05-09T08:20:31.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.400+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f9adc029070 con 0x7f9aec10e410 2024-05-09T08:20:31.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.401+0000 7f9af0a00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 0x7f9ad003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:31.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.401+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9adc01e070 con 0x7f9aec10e410 2024-05-09T08:20:31.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.402+0000 7f9aea000700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 0x7f9ad003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:31.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.403+0000 7f9aea000700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 0x7f9ad003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:31.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.407+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9adc013070 con 0x7f9aec10e410 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:31.607 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:31.608 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-09T08:20:31.608 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:31 smithi133 ceph-mon[29610]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:31.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:31.677+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) -- 0x7f9aec19f120 con 0x7f9ad00390f0 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:32.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:33 smithi133 ceph-mon[29610]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:33 smithi133 ceph-mon[29610]: Deploying cephadm binary to smithi188 2024-05-09T08:20:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:33 smithi133 ceph-mon[29610]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:35.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:35 smithi133 ceph-mon[29610]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:36.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.230+0000 7f9af0a00700 1 -- 172.21.15.133:0/3684229359 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9aec19f120 con 0x7f9ad00390f0 2024-05-09T08:20:36.232 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 msgr2=0x7f9ad003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 0x7f9ad003b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 msgr2=0x7f9aec19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006010 tx=0x7f9adc005030 comp rx=0 tx=0).stop 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 shutdown_connections 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9ad00390f0 0x7f9ad003b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 --2- 172.21.15.133:0/3684229359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec10e410 0x7f9aec19ddc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.233+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 >> 172.21.15.133:0/3684229359 conn(0x7f9aec077b80 msgr2=0x7f9aec0793d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.234+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 shutdown_connections 2024-05-09T08:20:36.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:36.234+0000 7f9af1e00700 1 -- 172.21.15.133:0/3684229359 wait complete. 2024-05-09T08:20:36.950 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T08:20:37.371 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:37.516 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:37.517 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:37 smithi133 ceph-mon[29610]: Added host smithi188 2024-05-09T08:20:37.517 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:37.517 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:37 smithi133 ceph-mon[29610]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:37.517 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:38.579 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 -- 172.21.15.133:0/4060928986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 msgr2=0x7fdcac102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 --2- 172.21.15.133:0/4060928986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac102360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009910 tx=0x7fdc9c009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 -- 172.21.15.133:0/4060928986 shutdown_connections 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 --2- 172.21.15.133:0/4060928986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac102360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 -- 172.21.15.133:0/4060928986 >> 172.21.15.133:0/4060928986 conn(0x7fdcac0fd830 msgr2=0x7fdcac0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 -- 172.21.15.133:0/4060928986 shutdown_connections 2024-05-09T08:20:39.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.369+0000 7fdcb2400700 1 -- 172.21.15.133:0/4060928986 wait complete. 2024-05-09T08:20:39.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.371+0000 7fdcb2400700 1 Processor -- start 2024-05-09T08:20:39.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.371+0000 7fdcb2400700 1 -- start start 2024-05-09T08:20:39.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.371+0000 7fdcb2400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:39.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.371+0000 7fdcb2400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac10c350 con 0x7fdcac101f80 2024-05-09T08:20:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.372+0000 7fdcab400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.372+0000 7fdcab400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43544/0 (socket says 172.21.15.133:43544) 2024-05-09T08:20:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.372+0000 7fdcab400700 1 -- 172.21.15.133:0/3984136851 learned_addr learned my addr 172.21.15.133:0/3984136851 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.372+0000 7fdcab400700 1 -- 172.21.15.133:0/3984136851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac19db30 con 0x7fdcac101f80 2024-05-09T08:20:39.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.372+0000 7fdcab400700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:39.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.373+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c026070 con 0x7fdcac101f80 2024-05-09T08:20:39.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.373+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c00bbd0 con 0x7fdcac101f80 2024-05-09T08:20:39.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.373+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac19de20 con 0x7fdcac101f80 2024-05-09T08:20:39.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.373+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc9c003910 con 0x7fdcac101f80 2024-05-09T08:20:39.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.373+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac197890 con 0x7fdcac101f80 2024-05-09T08:20:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.374+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fdc9c029070 con 0x7fdcac101f80 2024-05-09T08:20:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.374+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac103460 con 0x7fdcac101f80 2024-05-09T08:20:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.374+0000 7fdca3e00700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 0x7fdc9003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.374+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc9c01e070 con 0x7fdcac101f80 2024-05-09T08:20:39.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.375+0000 7fdcaaa00700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 0x7fdc9003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:39.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.375+0000 7fdcaaa00700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 0x7fdc9003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:39.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.380+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdc9c050030 con 0x7fdcac101f80 2024-05-09T08:20:39.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.606+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcac198c90 con 0x7fdc90039150 2024-05-09T08:20:39.608 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:39 smithi133 ceph-mon[29610]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:39.608 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:39.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.609+0000 7fdca3e00700 1 -- 172.21.15.133:0/3984136851 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fdcac198c90 con 0x7fdc90039150 2024-05-09T08:20:39.611 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:20:39.611 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-05-09T08:20:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 msgr2=0x7fdc9003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:39.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 0x7fdc9003b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 msgr2=0x7fdcac19d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c005030 comp rx=0 tx=0).stop 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 shutdown_connections 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdc90039150 0x7fdc9003b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 --2- 172.21.15.133:0/3984136851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcac101f80 0x7fdcac19d5f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 >> 172.21.15.133:0/3984136851 conn(0x7fdcac0fd830 msgr2=0x7fdcac103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:39.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 shutdown_connections 2024-05-09T08:20:39.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:39.612+0000 7fdcb2400700 1 -- 172.21.15.133:0/3984136851 wait complete. 2024-05-09T08:20:40.313 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T08:20:40.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-09T08:20:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:40 smithi133 ceph-mon[29610]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:20:40.735 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:20:41.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:41 smithi133 ceph-mon[29610]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:42.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 -- 172.21.15.133:0/2062874929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 msgr2=0x7eff3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:42.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 --2- 172.21.15.133:0/2062874929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c074aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009910 tx=0x7eff2c009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 -- 172.21.15.133:0/2062874929 shutdown_connections 2024-05-09T08:20:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 --2- 172.21.15.133:0/2062874929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c074aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 -- 172.21.15.133:0/2062874929 >> 172.21.15.133:0/2062874929 conn(0x7eff3c0fd810 msgr2=0x7eff3c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.719+0000 7eff42a00700 1 -- 172.21.15.133:0/2062874929 shutdown_connections 2024-05-09T08:20:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.720+0000 7eff42a00700 1 -- 172.21.15.133:0/2062874929 wait complete. 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.721+0000 7eff42a00700 1 Processor -- start 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.721+0000 7eff42a00700 1 -- start start 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.721+0000 7eff42a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.721+0000 7eff42a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c1129a0 con 0x7eff3c0746c0 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.722+0000 7eff3be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.722+0000 7eff3be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43550/0 (socket says 172.21.15.133:43550) 2024-05-09T08:20:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.722+0000 7eff3be00700 1 -- 172.21.15.133:0/3117830612 learned_addr learned my addr 172.21.15.133:0/3117830612 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:20:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.722+0000 7eff3be00700 1 -- 172.21.15.133:0/3117830612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c19e420 con 0x7eff3c0746c0 2024-05-09T08:20:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.722+0000 7eff3be00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.723+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff2c026070 con 0x7eff3c0746c0 2024-05-09T08:20:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.723+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2c00bc30 con 0x7eff3c0746c0 2024-05-09T08:20:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.723+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff2c0039a0 con 0x7eff3c0746c0 2024-05-09T08:20:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.723+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c19e6b0 con 0x7eff3c0746c0 2024-05-09T08:20:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.723+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c19eb10 con 0x7eff3c0746c0 2024-05-09T08:20:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.725+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c075e00 con 0x7eff3c0746c0 2024-05-09T08:20:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.725+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7eff2c029070 con 0x7eff3c0746c0 2024-05-09T08:20:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.725+0000 7eff39600700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 0x7eff2003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.725+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff2c01e070 con 0x7eff3c0746c0 2024-05-09T08:20:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.725+0000 7eff33e00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 0x7eff2003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.726+0000 7eff33e00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 0x7eff2003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff24006fd0 tx=0x7eff24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:42.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.731+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff2c013070 con 0x7eff3c0746c0 2024-05-09T08:20:42.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:42.959+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7eff3c19f240 con 0x7eff3c0746c0 2024-05-09T08:20:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.091+0000 7eff39600700 1 -- 172.21.15.133:0/3117830612 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7eff2c030090 con 0x7eff3c0746c0 2024-05-09T08:20:43.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 msgr2=0x7eff2003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:43.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 0x7eff2003b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7eff24006fd0 tx=0x7eff24006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:43.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 msgr2=0x7eff3c19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c005030 comp rx=0 tx=0).stop 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 shutdown_connections 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7eff20039150 0x7eff2003b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 --2- 172.21.15.133:0/3117830612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff3c0746c0 0x7eff3c19dee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 >> 172.21.15.133:0/3117830612 conn(0x7eff3c0fd810 msgr2=0x7eff3c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.096+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 shutdown_connections 2024-05-09T08:20:43.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:20:43.097+0000 7eff42a00700 1 -- 172.21.15.133:0/3117830612 wait complete. 2024-05-09T08:20:43.100 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-05-09T08:20:43.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:43.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:43 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3117830612' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T08:20:43.874 INFO:tasks.cephadm:Adding mon.a on smithi133 2024-05-09T08:20:43.874 INFO:tasks.cephadm:Adding mon.b on smithi160 2024-05-09T08:20:43.874 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-05-09T08:20:43.875 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c' 2024-05-09T08:20:44.101 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:44.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3117830612' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:44.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:45.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:45 smithi133 ceph-mon[29610]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:45.529 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 -- 172.21.15.188:0/953784679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 msgr2=0x7f8e78103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 --2- 172.21.15.188:0/953784679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78103320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800b0a0 tx=0x7f8e6800b3b0 comp rx=0 tx=0).stop 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 -- 172.21.15.188:0/953784679 shutdown_connections 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 --2- 172.21.15.188:0/953784679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78103320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 -- 172.21.15.188:0/953784679 >> 172.21.15.188:0/953784679 conn(0x7f8e780778f0 msgr2=0x7f8e780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 -- 172.21.15.188:0/953784679 shutdown_connections 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.526+0000 7f8e7d844700 1 -- 172.21.15.188:0/953784679 wait complete. 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.527+0000 7f8e7d844700 1 Processor -- start 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.527+0000 7f8e7d844700 1 -- start start 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.527+0000 7f8e7d844700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.527+0000 7f8e7d844700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7810c3a0 con 0x7f8e78102f40 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.528+0000 7f8e76ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:45.530 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.528+0000 7f8e76ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.188:43792/0 (socket says 172.21.15.188:43792) 2024-05-09T08:20:45.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.528+0000 7f8e76ffd700 1 -- 172.21.15.188:0/2510989708 learned_addr learned my addr 172.21.15.188:0/2510989708 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e76ffd700 1 -- 172.21.15.188:0/2510989708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e78102a90 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e76ffd700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e680062a0 tx=0x7f8e6800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e68017050 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e781010e0 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781015a0 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e68023070 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.529+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e6801e450 con 0x7f8e78102f40 2024-05-09T08:20:45.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.530+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e78069fb0 con 0x7f8e78102f40 2024-05-09T08:20:45.533 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.531+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8e68028460 con 0x7f8e78102f40 2024-05-09T08:20:45.533 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.531+0000 7f8e7c842700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 0x7f8e6403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:45.533 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.531+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8e6800f050 con 0x7f8e78102f40 2024-05-09T08:20:45.533 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.531+0000 7f8e767fc700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 0x7f8e6403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:45.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.532+0000 7f8e767fc700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 0x7f8e6403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e60006fd0 tx=0x7f8e60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:45.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.545+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e6801d3b0 con 0x7f8e78102f40 2024-05-09T08:20:45.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.750+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) -- 0x7f8e78102140 con 0x7f8e640390f0 2024-05-09T08:20:45.766 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.765+0000 7f8e7c842700 1 -- 172.21.15.188:0/2510989708 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8e78102140 con 0x7f8e640390f0 2024-05-09T08:20:45.766 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-05-09T08:20:45.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 msgr2=0x7f8e6403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:45.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 0x7f8e6403b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e60006fd0 tx=0x7f8e60006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:45.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 msgr2=0x7f8e78100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e680062a0 tx=0x7f8e6800bdd0 comp rx=0 tx=0).stop 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 shutdown_connections 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e640390f0 0x7f8e6403b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 --2- 172.21.15.188:0/2510989708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e78102f40 0x7f8e78100ba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.766+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 >> 172.21.15.188:0/2510989708 conn(0x7f8e780778f0 msgr2=0x7f8e78103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.767+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 shutdown_connections 2024-05-09T08:20:45.769 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:45.767+0000 7f8e7d844700 1 -- 172.21.15.188:0/2510989708 wait complete. 2024-05-09T08:20:46.341 DEBUG:teuthology.orchestra.run.smithi160:mon.b> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.b.service 2024-05-09T08:20:46.344 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.c.service 2024-05-09T08:20:46.346 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T08:20:46.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T08:20:47.068 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c;count:3 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:20:47.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:47.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: Deploying daemon mon.c on smithi188 2024-05-09T08:20:47.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:46 smithi133 ceph-mon[29610]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:48.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 -- 172.21.15.188:0/1405859487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 msgr2=0x7f33d4102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:48.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 --2- 172.21.15.188:0/1405859487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d4102350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009910 tx=0x7f33c8009c20 comp rx=0 tx=0).stop 2024-05-09T08:20:48.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 -- 172.21.15.188:0/1405859487 shutdown_connections 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 --2- 172.21.15.188:0/1405859487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d4102350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 -- 172.21.15.188:0/1405859487 >> 172.21.15.188:0/1405859487 conn(0x7f33d40fd840 msgr2=0x7f33d40ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 -- 172.21.15.188:0/1405859487 shutdown_connections 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.388+0000 7f33daed1700 1 -- 172.21.15.188:0/1405859487 wait complete. 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33daed1700 1 Processor -- start 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33daed1700 1 -- start start 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33daed1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33daed1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33d410c340 con 0x7f33d4101f70 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33d8c6d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33d8c6d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.188:43810/0 (socket says 172.21.15.188:43810) 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.389+0000 7f33d8c6d700 1 -- 172.21.15.188:0/4041727448 learned_addr learned my addr 172.21.15.188:0/4041727448 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33d8c6d700 1 -- 172.21.15.188:0/4041727448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d419daf0 con 0x7f33d4101f70 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33d8c6d700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33c8005850 tx=0x7f33c8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:48.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8026070 con 0x7f33d4101f70 2024-05-09T08:20:48.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d419dde0 con 0x7f33d4101f70 2024-05-09T08:20:48.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c800bbd0 con 0x7f33d4101f70 2024-05-09T08:20:48.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c8003970 con 0x7f33d4101f70 2024-05-09T08:20:48.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.390+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d4197850 con 0x7f33d4101f70 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.391+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d4103450 con 0x7f33d4101f70 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.392+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f33c8029070 con 0x7f33d4101f70 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.392+0000 7f33d1ffb700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 0x7f33c403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.392+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f33c801e070 con 0x7f33d4101f70 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.392+0000 7f33d3fff700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 0x7f33c403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:48.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.393+0000 7f33d3fff700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 0x7f33c403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:48.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.407+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f33c8013070 con 0x7f33d4101f70 2024-05-09T08:20:48.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.693+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f33d40008d0 con 0x7f33d4101f70 2024-05-09T08:20:48.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.694+0000 7f33d1ffb700 1 -- 172.21.15.188:0/4041727448 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f33c8030030 con 0x7f33d4101f70 2024-05-09T08:20:48.696 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:20:48.696 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","modified":"2024-05-09T08:18:44.970809Z","created":"2024-05-09T08:18:44.970809Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 msgr2=0x7f33c403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 0x7f33c403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006fd0 tx=0x7f33c0006e40 comp rx=0 tx=0).stop 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 msgr2=0x7f33d419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33c8005850 tx=0x7f33c8005030 comp rx=0 tx=0).stop 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 shutdown_connections 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f33c40390f0 0x7f33c403b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 --2- 172.21.15.188:0/4041727448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d4101f70 0x7f33d419d5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:48.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.696+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 >> 172.21.15.188:0/4041727448 conn(0x7f33d40fd840 msgr2=0x7f33d4103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:48.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.697+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 shutdown_connections 2024-05-09T08:20:48.699 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:48.697+0000 7f33daed1700 1 -- 172.21.15.188:0/4041727448 wait complete. 2024-05-09T08:20:48.699 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-05-09T08:20:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:49 smithi133 ceph-mon[29610]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:49 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/4041727448' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:20:50.278 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T08:20:50.278 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T08:20:50.581 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:20:51.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:51 smithi188 systemd[1]: Starting Ceph mon.c for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:20:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:51 smithi133 ceph-mon[29610]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.176 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:51 smithi188 podman[34602]: 2024-05-09 08:20:51.764421526 +0000 UTC m=+0.019101426 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:20:52.176 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:51 smithi188 podman[34602]: 2024-05-09 08:20:51.877502564 +0000 UTC m=+0.132182449 container create 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 podman[34602]: 2024-05-09 08:20:52.256876056 +0000 UTC m=+0.511555940 container init 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 podman[34602]: 2024-05-09 08:20:52.260312575 +0000 UTC m=+0.514992460 container start 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev), process ceph-mon, pid 2 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pidfile_write: ignore empty --pid-file 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: load: jerasure load: lrc 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: RocksDB version: 7.9.2 2024-05-09T08:20:52.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Git sha 0 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Compile date 2024-05-07 19:19:25 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: DB SUMMARY 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: DB Session ID: 2OGNB7M4RFCUOHV9C5S0 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: CURRENT file: CURRENT 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.error_if_exists: 0 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.create_if_missing: 0 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.paranoid_checks: 1 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T08:20:52.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.env: 0x560d8473dc60 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.info_log: 0x560d863ebac0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.statistics: (nil) 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.use_fsync: 0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_log_file_size: 0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T08:20:52.437 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_fallocate: 1 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.use_direct_reads: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.db_log_dir: 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.wal_dir: 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T08:20:52.438 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.write_buffer_manager: 0x560d863be1e0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.unordered_write: 0 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T08:20:52.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T08:20:52.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.row_cache: None 2024-05-09T08:20:52.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.wal_filter: None 2024-05-09T08:20:52.441 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T08:20:52.441 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T08:20:52.441 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.two_write_queues: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.wal_compression: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.atomic_flush: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.log_readahead_size: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T08:20:52.442 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_background_jobs: 2 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_background_compactions: -1 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_subcompactions: 1 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_open_files: -1 2024-05-09T08:20:52.443 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_background_flushes: -1 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Compression algorithms supported: 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kZSTD supported: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kXpressCompression supported: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kZlibCompression supported: 1 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kSnappyCompression supported: 1 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kLZ4Compression supported: 1 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: kBZip2Compression supported: 0 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T08:20:52.444 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.merge_operator: 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_filter: None 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560d863ebbc0) 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks: 1 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T08:20:52.445 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: pin_top_level_index_and_filter: 1 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_type: 0 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_index_type: 0 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_shortening: 1 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: checksum: 4 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: no_block_cache: 0 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache: 0x560d863ef090 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_name: BinnedLRUCache 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_options: 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: capacity : 536870912 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_shard_bits : 4 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: strict_capacity_limit : 0 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: high_pri_pool_ratio: 0.000 2024-05-09T08:20:52.446 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_cache_compressed: (nil) 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: persistent_cache: (nil) 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size: 4096 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_size_deviation: 10 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_restart_interval: 16 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: index_block_restart_interval: 1 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: metadata_block_size: 4096 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: partition_filters: 0 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: use_delta_encoding: 1 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: filter_policy: bloomfilter 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: whole_key_filtering: 1 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: verify_compression: 0 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: read_amp_bytes_per_bit: 0 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: format_version: 5 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_index_compression: 1 2024-05-09T08:20:52.447 INFO:journalctl@ceph.mon.c.smithi188.stdout: block_align: 0 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout: max_auto_readahead_size: 262144 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout: prepopulate_block_cache: 0 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout: initial_auto_readahead_size: 8192 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression: NoCompression 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.num_levels: 7 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T08:20:52.448 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T08:20:52.449 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T08:20:52.450 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T08:20:52.451 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.table_properties_collectors: 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.inplace_update_support: 0 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T08:20:52.452 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.bloom_locality: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.max_successive_merges: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.ttl: 2592000 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enable_blob_files: false 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.min_blob_size: 0 2024-05-09T08:20:52.453 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b705b629-c306-47b7-8735-6213f3ce3e67 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242852290022, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T08:20:52.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242852290441, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715242852, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b705b629-c306-47b7-8735-6213f3ce3e67", "db_session_id": "2OGNB7M4RFCUOHV9C5S0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715242852290519, "job": 1, "event": "recovery_finished"} 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560d864e0000 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: DB pointer 0x560d864ca000 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** DB Stats ** 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:20:52.455 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** Compaction Stats [default] ** 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T08:20:52.456 INFO:journalctl@ceph.mon.c.smithi188.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache BinnedLRUCache@0x560d863ef090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T08:20:52.457 INFO:journalctl@ceph.mon.c.smithi188.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: using public_addr v2:172.21.15.188:0/0 -> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: starting mon.c rank -1 at public addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] at bind addrs [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(???) e0 preinit fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: e1 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: legacy client fscid: -1 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: 2024-05-09T08:20:52.458 INFO:journalctl@ceph.mon.c.smithi188.stdout: No filesystems configured 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Added host smithi160 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:52.459 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:52.460 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Deploying cephadm binary to smithi188 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Added host smithi188 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:52.461 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:52.462 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3117830612' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3117830612' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c;count:3 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.463 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: Deploying daemon mon.c on smithi188 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/4041727448' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 ceph-mon[34645]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 bash[34602]: 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 2024-05-09T08:20:52.464 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:52 smithi188 systemd[1]: Started Ceph mon.c for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:20:57.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:20:57.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: mon.a calling monitor election 2024-05-09T08:20:57.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: mon.c calling monitor election 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: monmap epoch 2 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:57.682 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: last_changed 2024-05-09T08:20:52.321547+0000 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: min_mon_release 19 (squid) 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: election_strategy: 1 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: fsmap 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: mgrmap e13: x(active, since 60s) 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:20:57.683 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:57.684 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:57.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:20:57.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: mon.a calling monitor election 2024-05-09T08:20:57.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: mon.c calling monitor election 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: monmap epoch 2 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: last_changed 2024-05-09T08:20:52.321547+0000 2024-05-09T08:20:57.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: min_mon_release 19 (squid) 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: election_strategy: 1 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: fsmap 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: mgrmap e13: x(active, since 60s) 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:57.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3023080795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49f0006230 msgr2=0x7f49f00086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3023080795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49f0006230 0x7f49f00086a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4a0806a730 tx=0x7f49f800fe90 comp rx=0 tx=0).stop 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3023080795 shutdown_connections 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3023080795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49f0006230 0x7f49f00086a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3023080795 >> 172.21.15.188:0/3023080795 conn(0x7f4a080fc830 msgr2=0x7f4a080fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3023080795 shutdown_connections 2024-05-09T08:20:58.565 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.561+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3023080795 wait complete. 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.562+0000 7f4a0dba0700 1 Processor -- start 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a0dba0700 1 -- start start 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a0dba0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a0dba0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 0x7f4a08197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a0dba0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08109eb0 con 0x7f4a0819dc80 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a0dba0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08109d30 con 0x7f49f0006230 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41014/0 (socket says 172.21.15.188:41014) 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 -- 172.21.15.188:0/3423003596 learned_addr learned my addr 172.21.15.188:0/3423003596 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 -- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 msgr2=0x7f4a0819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 -- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 msgr2=0x7f4a0819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:20:58.566 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.563+0000 7f4a077fe700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:20:58.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.564+0000 7f4a06ffd700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 0x7f4a08197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.564+0000 7f4a06ffd700 1 -- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 msgr2=0x7f4a0819d740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.564+0000 7f4a06ffd700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.564+0000 7f4a06ffd700 1 -- 172.21.15.188:0/3423003596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0819e2c0 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a06ffd700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 0x7f4a08197900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b4d0 tx=0x7f49fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f49fc011630 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a08197e40 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a081983d0 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc011cb0 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.565+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f49fc0193b0 con 0x7f4a0819dc80 2024-05-09T08:20:58.568 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.566+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f0001fe0 con 0x7f4a0819dc80 2024-05-09T08:20:58.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.567+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f49fc018020 con 0x7f4a0819dc80 2024-05-09T08:20:58.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.567+0000 7f4a04ff9700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 0x7f49f403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:20:58.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.567+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f49fc04bab0 con 0x7f4a0819dc80 2024-05-09T08:20:58.569 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.567+0000 7f4a077fe700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 0x7f49f403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:20:58.570 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.568+0000 7f4a077fe700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 0x7f49f403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005b40 tx=0x7f49f8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:20:58.584 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.583+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f49fc01f020 con 0x7f4a0819dc80 2024-05-09T08:20:58.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:58.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:20:58.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:58.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:58 smithi133 ceph-mon[29610]: Deploying daemon mon.b on smithi160 2024-05-09T08:20:58.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:58.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:20:58.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:20:58.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:20:58.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:58 smithi188 ceph-mon[34645]: Deploying daemon mon.b on smithi160 2024-05-09T08:20:58.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:20:58.863 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.862+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f49f00032b0 con 0x7f4a0819dc80 2024-05-09T08:20:58.864 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.863+0000 7f4a04ff9700 1 -- 172.21.15.188:0/3423003596 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f49fc018570 con 0x7f4a0819dc80 2024-05-09T08:20:58.865 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:20:58.865 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","modified":"2024-05-09T08:20:52.321547Z","created":"2024-05-09T08:18:44.970809Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 msgr2=0x7f49f403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 0x7f49f403b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005b40 tx=0x7f49f8005a90 comp rx=0 tx=0).stop 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 msgr2=0x7f4a08197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 0x7f4a08197900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b4d0 tx=0x7f49fc00b890 comp rx=0 tx=0).stop 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 shutdown_connections 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f49f4038e90 0x7f49f403b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f0006230 0x7f4a0819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 --2- 172.21.15.188:0/3423003596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0819dc80 0x7f4a08197900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 >> 172.21.15.188:0/3423003596 conn(0x7f4a080fc830 msgr2=0x7f4a08103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 shutdown_connections 2024-05-09T08:20:58.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:20:58.865+0000 7f4a0dba0700 1 -- 172.21.15.188:0/3423003596 wait complete. 2024-05-09T08:20:58.868 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-05-09T08:20:59.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:59 smithi188 ceph-mon[34645]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:59.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:20:59 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3423003596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:20:59.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:59 smithi133 ceph-mon[29610]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:20:59.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:20:59 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3423003596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:21:00.457 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T08:21:00.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-09T08:21:00.700 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:21:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: Deploying daemon mon.b on smithi160 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:21:01.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:01.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3423003596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:21:01.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 ceph-mon[34070]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T08:21:01.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 bash[34050]: a1c25f7a82775c3557c2e0d1ed059654e6a8d0f157fe8731e3da41348e652975 2024-05-09T08:21:01.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:01 smithi160 systemd[1]: Started Ceph mon.b for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 -- 172.21.15.188:0/1068023038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14840035c0 msgr2=0x7f1484005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 --2- 172.21.15.188:0/1068023038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14840035c0 0x7f1484005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1498078450 tx=0x7f1490009e80 comp rx=0 tx=0).stop 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 -- 172.21.15.188:0/1068023038 shutdown_connections 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 --2- 172.21.15.188:0/1068023038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14840035c0 0x7f1484005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 -- 172.21.15.188:0/1068023038 >> 172.21.15.188:0/1068023038 conn(0x7f1498077160 msgr2=0x7f1498077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 -- 172.21.15.188:0/1068023038 shutdown_connections 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.542+0000 7f149f21c700 1 -- 172.21.15.188:0/1068023038 wait complete. 2024-05-09T08:21:06.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.543+0000 7f149f21c700 1 Processor -- start 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.543+0000 7f149f21c700 1 -- start start 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.543+0000 7f149f21c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149f21c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149f21c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 0x7f14981982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149f21c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1498109e60 con 0x7f1498197e20 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149f21c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1498109ce0 con 0x7f149819dc60 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149f21c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1498109fe0 con 0x7f14840035c0 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57130/0 (socket says 172.21.15.188:57130) 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 -- 172.21.15.188:0/3127388323 learned_addr learned my addr 172.21.15.188:0/3127388323 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f1497fff700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:06.547 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 0x7f14981982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 msgr2=0x7f149819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 msgr2=0x7f149819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149cfb8700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f1497fff700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 msgr2=0x7f14981978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f1497fff700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 msgr2=0x7f14981978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f1497fff700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f1497fff700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 msgr2=0x7f149819d720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 msgr2=0x7f14981978e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.544+0000 7f149d7b9700 1 -- 172.21.15.188:0/3127388323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14981a45f0 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f149d7b9700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 0x7f14981982d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f148800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488015020 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14981a4820 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1488011d10 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148801ae10 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.545+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14981a4d60 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.547+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1488018020 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.547+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1498063980 con 0x7f1498197e20 2024-05-09T08:21:06.550 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.547+0000 7f1495ffb700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 0x7f147803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:06.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.547+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f148804bb70 con 0x7f1498197e20 2024-05-09T08:21:06.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.547+0000 7f149cfb8700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 0x7f147803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:06.551 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.548+0000 7f149cfb8700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 0x7f147803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1490009a30 tx=0x7f149000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:06.555 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.554+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f148801f020 con 0x7f1498197e20 2024-05-09T08:21:06.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.845+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f149802db80 con 0x7f1498197e20 2024-05-09T08:21:06.847 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: mon.c calling monitor election 2024-05-09T08:21:06.847 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:21:06.847 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: mon.a calling monitor election 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: monmap epoch 3 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:06.848 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: min_mon_release 19 (squid) 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: election_strategy: 1 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: fsmap 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: mgrmap e13: x(active, since 69s) 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:06.849 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:06.850 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.846+0000 7f1495ffb700 1 -- 172.21.15.188:0/3127388323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f1488018570 con 0x7f1498197e20 2024-05-09T08:21:06.850 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T08:21:06.850 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","modified":"2024-05-09T08:21:01.415621Z","created":"2024-05-09T08:18:44.970809Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 msgr2=0x7f147803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 0x7f147803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1490009a30 tx=0x7f149000b040 comp rx=0 tx=0).stop 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 msgr2=0x7f14981982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 0x7f14981982d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f148800b7e0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 shutdown_connections 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1498197e20 0x7f14981982d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1478039580 0x7f147803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14840035c0 0x7f149819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.851 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 --2- 172.21.15.188:0/3127388323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f149819dc60 0x7f14981978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:06.852 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 >> 172.21.15.188:0/3127388323 conn(0x7f1498077160 msgr2=0x7f14980fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:06.852 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.848+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 shutdown_connections 2024-05-09T08:21:06.852 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:06.849+0000 7f149f21c700 1 -- 172.21.15.188:0/3127388323 wait complete. 2024-05-09T08:21:06.852 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-05-09T08:21:06.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: mon.c calling monitor election 2024-05-09T08:21:06.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:21:06.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: mon.a calling monitor election 2024-05-09T08:21:06.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: monmap epoch 3 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: min_mon_release 19 (squid) 2024-05-09T08:21:06.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: election_strategy: 1 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: fsmap 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: mgrmap e13: x(active, since 69s) 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:06.932 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:07.556 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:07.557 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:07.557 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:07.557 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:07.557 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3127388323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:21:07.557 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:07.582 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T08:21:07.582 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3127388323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:21:07.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:07.984 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:21:08.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: mon.c calling monitor election 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: mon.a calling monitor election 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: monmap epoch 3 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: min_mon_release 19 (squid) 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: election_strategy: 1 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: fsmap 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: mgrmap e13: x(active, since 69s) 2024-05-09T08:21:08.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3127388323' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T08:21:08.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mon.b calling monitor election 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: Updating smithi133:/etc/ceph/ceph.conf 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mon.b calling monitor election 2024-05-09T08:21:09.600 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mon.c calling monitor election 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mon.a calling monitor election 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: monmap epoch 3 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:09.601 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: min_mon_release 19 (squid) 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: election_strategy: 1 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: fsmap 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: mgrmap e13: x(active, since 71s) 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:09.602 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.603 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.603 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.603 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.603 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.603 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.637 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mon.b calling monitor election 2024-05-09T08:21:09.637 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: Updating smithi133:/etc/ceph/ceph.conf 2024-05-09T08:21:09.637 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-09T08:21:09.637 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-09T08:21:09.637 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mon.b calling monitor election 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mon.c calling monitor election 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mon.a calling monitor election 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: monmap epoch 3 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:09.638 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: min_mon_release 19 (squid) 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: election_strategy: 1 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: fsmap 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: mgrmap e13: x(active, since 71s) 2024-05-09T08:21:09.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.640 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.641 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mon.b calling monitor election 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: Updating smithi133:/etc/ceph/ceph.conf 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: Updating smithi160:/etc/ceph/ceph.conf 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: Updating smithi188:/etc/ceph/ceph.conf 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mon.b calling monitor election 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mon.c calling monitor election 2024-05-09T08:21:09.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mon.a calling monitor election 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: monmap epoch 3 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: last_changed 2024-05-09T08:21:01.415621+0000 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: created 2024-05-09T08:18:44.970809+0000 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: min_mon_release 19 (squid) 2024-05-09T08:21:09.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: election_strategy: 1 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: 0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: 2: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: fsmap 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T08:21:09.667 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: mgrmap e13: x(active, since 71s) 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: Updating smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: Updating smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:09.668 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 -- 172.21.15.133:0/1035721738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1814100e60 msgr2=0x7f1814101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 --2- 172.21.15.133:0/1035721738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1814100e60 0x7f1814101240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1808009910 tx=0x7f1808009c20 comp rx=0 tx=0).stop 2024-05-09T08:21:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 -- 172.21.15.133:0/1035721738 shutdown_connections 2024-05-09T08:21:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 --2- 172.21.15.133:0/1035721738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1814100e60 0x7f1814101240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 -- 172.21.15.133:0/1035721738 >> 172.21.15.133:0/1035721738 conn(0x7f18140778f0 msgr2=0x7f18140feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 -- 172.21.15.133:0/1035721738 shutdown_connections 2024-05-09T08:21:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.237+0000 7f181b400700 1 -- 172.21.15.133:0/1035721738 wait complete. 2024-05-09T08:21:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.238+0000 7f181b400700 1 Processor -- start 2024-05-09T08:21:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 -- start start 2024-05-09T08:21:10.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 0x7f181419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 0x7f1814199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 0x7f181419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f181410b230 con 0x7f181419fe60 2024-05-09T08:21:10.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f181410b0b0 con 0x7f181419a020 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.239+0000 7f181b400700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f181410b3b0 con 0x7f1814100e60 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1813e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 0x7f1814199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1813e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 0x7f1814199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50038/0 (socket says 172.21.15.133:50038) 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1813e00700 1 -- 172.21.15.133:0/26323558 learned_addr learned my addr 172.21.15.133:0/26323558 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 0x7f181419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:10.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1819400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 0x7f181419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 -- 172.21.15.133:0/26323558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 msgr2=0x7f181419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 0x7f181419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 -- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 msgr2=0x7f1814199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 0x7f1814199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1818a00700 1 -- 172.21.15.133:0/26323558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18141a67f0 con 0x7f1814100e60 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.240+0000 7f1819400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 0x7f181419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:21:10.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f1818a00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 0x7f181419f920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1808005950 tx=0x7f1808004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808026070 con 0x7f1814100e60 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180800b6f0 con 0x7f1814100e60 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18141a6a20 con 0x7f1814100e60 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080038e0 con 0x7f1814100e60 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.241+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a6f60 con 0x7f1814100e60 2024-05-09T08:21:10.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.242+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1808029070 con 0x7f1814100e60 2024-05-09T08:21:10.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.243+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1814102340 con 0x7f1814100e60 2024-05-09T08:21:10.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.243+0000 7f1811600700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 0x7f17fc03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:10.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.243+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f180804cdc0 con 0x7f1814100e60 2024-05-09T08:21:10.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.244+0000 7f1813e00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 0x7f17fc03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:10.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.244+0000 7f1813e00700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 0x7f17fc03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1800009b80 tx=0x7f18000092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:10.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.248+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1808013070 con 0x7f1814100e60 2024-05-09T08:21:10.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.478+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f18141a7190 con 0x7f1814100e60 2024-05-09T08:21:10.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.478+0000 7f1811600700 1 -- 172.21.15.133:0/26323558 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f1808030020 con 0x7f1814100e60 2024-05-09T08:21:10.480 INFO:teuthology.orchestra.run.smithi133.stdout:# minimal ceph.conf for 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:10.480 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-05-09T08:21:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:21:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: mon_host = [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-05-09T08:21:10.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 msgr2=0x7f17fc03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:10.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 0x7f17fc03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1800009b80 tx=0x7f18000092c0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 msgr2=0x7f181419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:10.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 0x7f181419f920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1808005950 tx=0x7f1808004f80 comp rx=0 tx=0).stop 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 shutdown_connections 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f181419a020 0x7f181419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17fc039580 0x7f17fc03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1814100e60 0x7f181419f920 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 --2- 172.21.15.133:0/26323558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f181419fe60 0x7f1814199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.483+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 >> 172.21.15.133:0/26323558 conn(0x7f18140778f0 msgr2=0x7f1814069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.484+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 shutdown_connections 2024-05-09T08:21:10.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:10.484+0000 7f181b400700 1 -- 172.21.15.133:0/26323558 wait complete. 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: Updating smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.504 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.504 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:10.504 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:10.504 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:10.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: Updating smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:10.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:10.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:10.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: Updating smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:10.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:11.200 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T08:21:11.200 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:21:11.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T08:21:11.278 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:21:11.278 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:21:11.396 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:21:11.397 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T08:21:11.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:11 smithi133 ceph-mon[29610]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T08:21:11.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:11 smithi133 ceph-mon[29610]: Reconfiguring daemon mon.a on smithi133 2024-05-09T08:21:11.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:11 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/26323558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:11.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:11 smithi133 ceph-mon[29610]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:11.431 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:21:11.431 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:21:11.500 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:21:11.500 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T08:21:11.532 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:21:11.532 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T08:21:11.600 INFO:tasks.cephadm:Adding mgr.x on smithi133 2024-05-09T08:21:11.600 INFO:tasks.cephadm:Adding mgr.y on smithi160 2024-05-09T08:21:11.600 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-05-09T08:21:11.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi133=x;smithi160=y;smithi188=z' 2024-05-09T08:21:11.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:11 smithi188 ceph-mon[34645]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T08:21:11.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:11 smithi188 ceph-mon[34645]: Reconfiguring daemon mon.a on smithi133 2024-05-09T08:21:11.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:11 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/26323558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:11.663 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:11 smithi188 ceph-mon[34645]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:11.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:11 smithi160 ceph-mon[34070]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T08:21:11.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:11 smithi160 ceph-mon[34070]: Reconfiguring daemon mon.a on smithi133 2024-05-09T08:21:11.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:11 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/26323558' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:11.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:11 smithi160 ceph-mon[34070]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:11.873 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 -- 172.21.15.188:0/942260875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c100d90 msgr2=0x7f138c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 --2- 172.21.15.188:0/942260875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c100d90 0x7f138c1011b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f138800b0a0 tx=0x7f138800b3b0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 -- 172.21.15.188:0/942260875 shutdown_connections 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 --2- 172.21.15.188:0/942260875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c100d90 0x7f138c1011b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 -- 172.21.15.188:0/942260875 >> 172.21.15.188:0/942260875 conn(0x7f138c0fc330 msgr2=0x7f138c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.279+0000 7f13934d5700 1 -- 172.21.15.188:0/942260875 shutdown_connections 2024-05-09T08:21:13.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 -- 172.21.15.188:0/942260875 wait complete. 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 Processor -- start 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 -- start start 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c0fc920 0x7f138c0fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f138c0fd2e0 0x7f138c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f138c106dd0 con 0x7f138c0fc920 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f138c106c50 con 0x7f138c0fd2e0 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f13934d5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f138c106f50 con 0x7f138c100d90 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f1391271700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f1391271700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57146/0 (socket says 172.21.15.188:57146) 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.280+0000 7f1391271700 1 -- 172.21.15.188:0/3151806820 learned_addr learned my addr 172.21.15.188:0/3151806820 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 -- 172.21.15.188:0/3151806820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f138c0fd2e0 msgr2=0x7f138c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:13.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f138c0fd2e0 0x7f138c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 -- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c0fc920 msgr2=0x7f138c0fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c0fc920 0x7f138c0fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 -- 172.21.15.188:0/3151806820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c1a1ce0 con 0x7f138c100d90 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f1391271700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f13880095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388017040 con 0x7f138c100d90 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13880055b0 con 0x7f138c100d90 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1a1f70 con 0x7f138c100d90 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388012ba0 con 0x7f138c100d90 2024-05-09T08:21:13.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.281+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c1a2520 con 0x7f138c100d90 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.282+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f138801b420 con 0x7f138c100d90 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.282+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c063980 con 0x7f138c100d90 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.282+0000 7f13827fc700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 0x7f137803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.282+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f138800f020 con 0x7f138c100d90 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.283+0000 7f1390a70700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 0x7f137803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:13.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.284+0000 7f1390a70700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 0x7f137803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f137c0098e0 tx=0x7f137c006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:13.288 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.287+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1388029020 con 0x7f138c100d90 2024-05-09T08:21:13.493 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.492+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi160=y;smithi188=z", "target": ["mon-mgr", ""]}) -- 0x7f138c062760 con 0x7f1378039580 2024-05-09T08:21:13.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.506+0000 7f13827fc700 1 -- 172.21.15.188:0/3151806820 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f138c062760 con 0x7f1378039580 2024-05-09T08:21:13.507 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 msgr2=0x7f137803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 0x7f137803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f137c0098e0 tx=0x7f137c006dd0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 msgr2=0x7f138c0fe2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f80 tx=0x7f13880095c0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 shutdown_connections 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f138c0fd2e0 0x7f138c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1378039580 0x7f137803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138c100d90 0x7f138c0fe2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 --2- 172.21.15.188:0/3151806820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f138c0fc920 0x7f138c0fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 >> 172.21.15.188:0/3151806820 conn(0x7f138c0fc330 msgr2=0x7f138c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 shutdown_connections 2024-05-09T08:21:13.510 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:21:13.508+0000 7f13934d5700 1 -- 172.21.15.188:0/3151806820 wait complete. 2024-05-09T08:21:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: Reconfiguring mon.b (monmap changed)... 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: Reconfiguring daemon mon.b on smithi160 2024-05-09T08:21:14.059 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:14.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.060 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: Reconfiguring mon.b (monmap changed)... 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: Reconfiguring daemon mon.b on smithi160 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: Reconfiguring mon.b (monmap changed)... 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: Reconfiguring daemon mon.b on smithi160 2024-05-09T08:21:14.062 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.063 DEBUG:teuthology.orchestra.run.smithi160:mgr.y> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.y.service 2024-05-09T08:21:14.092 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.z.service 2024-05-09T08:21:14.112 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T08:21:14.112 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:21:14.112 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:21:14.170 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:21:14.170 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-05-09T08:21:14.219 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-05-09 08:20:13.092722667 +0000 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-05-09 08:14:58.769196560 +0000 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-05-09 08:14:58.769196560 +0000 2024-05-09T08:21:14.220 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-05-09 08:14:58.769196560 +0000 2024-05-09T08:21:14.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:21:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-05-09T08:21:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-05-09T08:21:14.357 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000393116 s, 1.3 MB/s 2024-05-09T08:21:14.360 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:21:14.421 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-05-09 08:20:13.144721721 +0000 2024-05-09T08:21:14.469 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-05-09 08:14:59.370185551 +0000 2024-05-09T08:21:14.470 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-05-09 08:14:59.370185551 +0000 2024-05-09T08:21:14.470 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-05-09 08:14:59.370185551 +0000 2024-05-09T08:21:14.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:21:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-05-09T08:21:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-05-09T08:21:14.539 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.00034751 s, 1.5 MB/s 2024-05-09T08:21:14.542 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:21:14.598 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-05-09T08:21:14.647 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:21:14.647 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:14.647 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-09T08:21:14.647 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:14.648 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:14.648 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-05-09 08:20:13.191720866 +0000 2024-05-09T08:21:14.648 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-05-09 08:14:59.959174761 +0000 2024-05-09T08:21:14.648 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-05-09 08:14:59.959174761 +0000 2024-05-09T08:21:14.648 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-05-09 08:14:59.959174761 +0000 2024-05-09T08:21:14.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi160=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: Saving service mgr spec with placement smithi133=x;smithi160=y;smithi188=z;count:3 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-05-09T08:21:14.725 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-05-09T08:21:14.725 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000345466 s, 1.5 MB/s 2024-05-09T08:21:14.729 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:21:14.829 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-05-09 08:20:13.240719975 +0000 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-05-09 08:15:00.561163733 +0000 2024-05-09T08:21:14.880 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-05-09 08:15:00.561163733 +0000 2024-05-09T08:21:14.881 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-05-09 08:15:00.561163733 +0000 2024-05-09T08:21:14.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi160=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: Saving service mgr spec with placement smithi133=x;smithi160=y;smithi188=z;count:3 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.884 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi160=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:14.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: Saving service mgr spec with placement smithi133=x;smithi160=y;smithi188=z;count:3 2024-05-09T08:21:14.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:14.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T08:21:14.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T08:21:14.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:14.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-05-09T08:21:14.958 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-05-09T08:21:14.958 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000393079 s, 1.3 MB/s 2024-05-09T08:21:14.962 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:21:15.017 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T08:21:15.017 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:21:15.037 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:21:15.037 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-09T08:21:15.095 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:21:15.095 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:15.095 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 08:21:00.870179453 +0000 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 08:14:05.885327009 +0000 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 08:14:05.885327009 +0000 2024-05-09T08:21:15.096 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 08:14:05.885327009 +0000 2024-05-09T08:21:15.096 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:21:15.161 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T08:21:15.161 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T08:21:15.161 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000269666 s, 1.9 MB/s 2024-05-09T08:21:15.163 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:21:15.221 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 08:21:00.871179439 +0000 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 08:14:06.243321591 +0000 2024-05-09T08:21:15.278 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 08:14:06.243321591 +0000 2024-05-09T08:21:15.279 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 08:14:06.243321591 +0000 2024-05-09T08:21:15.279 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:21:15.342 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T08:21:15.342 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T08:21:15.342 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000212622 s, 2.4 MB/s 2024-05-09T08:21:15.344 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:21:15.401 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-09T08:21:15.457 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:21:15.457 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 08:21:00.871179439 +0000 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 08:14:06.582316462 +0000 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 08:14:06.582316462 +0000 2024-05-09T08:21:15.458 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 08:14:06.582316462 +0000 2024-05-09T08:21:15.458 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:21:15.521 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T08:21:15.521 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T08:21:15.521 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000198009 s, 2.6 MB/s 2024-05-09T08:21:15.522 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:21:15.579 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-09T08:21:15.636 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 08:21:00.871179439 +0000 2024-05-09T08:21:15.637 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 08:14:06.922311318 +0000 2024-05-09T08:21:15.638 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 08:14:06.922311318 +0000 2024-05-09T08:21:15.638 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 08:14:06.922311318 +0000 2024-05-09T08:21:15.638 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:21:15.700 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T08:21:15.701 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T08:21:15.701 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000202281 s, 2.5 MB/s 2024-05-09T08:21:15.702 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:21:15.758 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T08:21:15.758 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:21:15.774 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:21:15.775 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:15.832 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 08:20:51.152091862 +0000 2024-05-09T08:21:15.833 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 08:13:55.071856108 +0000 2024-05-09T08:21:15.833 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 08:13:55.071856108 +0000 2024-05-09T08:21:15.833 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 08:13:55.071856108 +0000 2024-05-09T08:21:15.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:21:15.893 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:15 smithi188 ceph-mon[34645]: Reconfiguring mon.c (monmap changed)... 2024-05-09T08:21:15.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:15 smithi188 ceph-mon[34645]: Reconfiguring daemon mon.c on smithi188 2024-05-09T08:21:15.894 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:15 smithi188 ceph-mon[34645]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:15.896 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T08:21:15.896 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T08:21:15.896 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000188968 s, 2.7 MB/s 2024-05-09T08:21:15.898 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:21:15.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:15 smithi160 ceph-mon[34070]: Reconfiguring mon.c (monmap changed)... 2024-05-09T08:21:15.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:15 smithi160 ceph-mon[34070]: Reconfiguring daemon mon.c on smithi188 2024-05-09T08:21:15.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:15 smithi160 ceph-mon[34070]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:15.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:15 smithi133 ceph-mon[29610]: Reconfiguring mon.c (monmap changed)... 2024-05-09T08:21:15.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:15 smithi133 ceph-mon[29610]: Reconfiguring daemon mon.c on smithi188 2024-05-09T08:21:15.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:15 smithi133 ceph-mon[29610]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:15.954 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 08:20:51.153091843 +0000 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 08:13:55.409849705 +0000 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 08:13:55.409849705 +0000 2024-05-09T08:21:16.009 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 08:13:55.409849705 +0000 2024-05-09T08:21:16.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:21:16.071 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T08:21:16.071 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T08:21:16.071 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000199992 s, 2.6 MB/s 2024-05-09T08:21:16.073 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:21:16.129 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 08:20:51.153091843 +0000 2024-05-09T08:21:16.184 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 08:13:55.735843529 +0000 2024-05-09T08:21:16.185 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 08:13:55.735843529 +0000 2024-05-09T08:21:16.185 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 08:13:55.735843529 +0000 2024-05-09T08:21:16.185 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:21:16.248 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T08:21:16.248 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T08:21:16.248 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000191374 s, 2.7 MB/s 2024-05-09T08:21:16.249 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:21:16.304 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-05-09T08:21:16.358 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 08:20:51.153091843 +0000 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 08:13:56.067837239 +0000 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 08:13:56.067837239 +0000 2024-05-09T08:21:16.359 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 08:13:56.067837239 +0000 2024-05-09T08:21:16.359 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:21:16.421 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T08:21:16.421 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T08:21:16.421 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000179488 s, 2.9 MB/s 2024-05-09T08:21:16.422 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:21:16.478 INFO:tasks.cephadm:Deploying osd.0 on smithi133 with /dev/vg_nvme/lv_4... 2024-05-09T08:21:16.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T08:21:16.902 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:21:16.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:16.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:16.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:16.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.916 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:16 smithi160 ceph-mon[34070]: Deploying daemon mgr.z on smithi188 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:16 smithi133 ceph-mon[29610]: Deploying daemon mgr.z on smithi188 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:16.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:16.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:16.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:16.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:16.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:16 smithi188 ceph-mon[34645]: Deploying daemon mgr.z on smithi188 2024-05-09T08:21:17.875 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:17 smithi188 ceph-mon[34645]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:17.875 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:17 smithi188 systemd[1]: Starting Ceph mgr.z for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:21:17.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:17 smithi160 ceph-mon[34070]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:17.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:17 smithi133 ceph-mon[29610]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:17 smithi188 podman[36361]: 2024-05-09 08:21:17.7776389 +0000 UTC m=+0.018236650 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:17 smithi188 podman[36361]: 2024-05-09 08:21:17.878190696 +0000 UTC m=+0.118788472 container create 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 podman[36361]: 2024-05-09 08:21:18.080526005 +0000 UTC m=+0.321123738 container init 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc) 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 podman[36361]: 2024-05-09 08:21:18.082995158 +0000 UTC m=+0.323592894 container start 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-mgr[36380]: -- 172.21.15.188:0/3907541568 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55def23425a0 con 0x55def2316c00 2024-05-09T08:21:18.157 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 bash[36361]: 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d 2024-05-09T08:21:18.429 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 systemd[1]: Started Ceph mgr.z for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:21:18.429 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:18.241+0000 7f7743b32200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:21:18.429 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:18.427+0000 7f7743b32200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:21:18.686 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:18.529+0000 7f7743b32200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:21:19.136 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:18.837+0000 7f7743b32200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:21:19.136 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:18.920+0000 7f7743b32200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:21:19.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: Deploying daemon mgr.y on smithi160 2024-05-09T08:21:19.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:19 smithi188 ceph-mon[34645]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:19.653 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: Deploying daemon mgr.y on smithi160 2024-05-09T08:21:19.654 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:19 smithi133 ceph-mon[29610]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:19.654 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T08:21:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T08:21:19.655 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-05-09T08:21:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-05-09T08:21:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0370001 s, 283 MB/s 2024-05-09T08:21:19.655 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:19.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:19.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: Deploying daemon mgr.y on smithi160 2024-05-09T08:21:19.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:19 smithi160 ceph-mon[34070]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:20.320 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:20.152+0000 7f7743b32200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:21:20.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-05-09T08:21:20.414 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 systemd[1]: Starting Ceph mgr.y for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:21:20.684 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:20.318+0000 7f7743b32200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:21:20.684 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:20.491+0000 7f7743b32200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:21:20.693 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 podman[35456]: 2024-05-09 08:21:20.439741056 +0000 UTC m=+0.103541027 container create 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T08:21:20.693 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 podman[35456]: 2024-05-09 08:21:20.353878467 +0000 UTC m=+0.017678396 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:21:20.693 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 podman[35456]: 2024-05-09 08:21:20.61728252 +0000 UTC m=+0.281082423 container init 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:21:20.694 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 podman[35456]: 2024-05-09 08:21:20.619890371 +0000 UTC m=+0.283690274 container start 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T08:21:20.694 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 ceph-mgr[35476]: -- 172.21.15.160:0/1003302745 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a29e805a0 con 0x561a29e55000 2024-05-09T08:21:20.694 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 bash[35456]: 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa 2024-05-09T08:21:20.848 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:21:20.980 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 systemd[1]: Started Ceph mgr.y for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:21:20.980 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:20.791+0000 7f73139ac200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:21:20.980 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:20.978+0000 7f73139ac200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:21:21.076 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:20.776+0000 7f7743b32200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:21:21.076 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:20.868+0000 7f7743b32200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:21:21.077 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:21.075+0000 7f7743b32200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:21:21.409 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:21.082+0000 7f73139ac200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:21:21.410 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:21.407+0000 7f73139ac200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:21:21.434 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:21.276+0000 7f7743b32200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:21:21.664 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:21.496+0000 7f73139ac200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:21:21.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:21.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:21.876 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:21.876 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.877 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.877 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.877 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.877 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:21.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:21.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:22.184 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:21.912+0000 7f7743b32200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:21:22.613 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:22.354+0000 7f7743b32200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:21:22.613 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:22.452+0000 7f7743b32200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:21:22.761 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:22.762 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.914 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:22.759+0000 7f73139ac200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:21:22.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:22.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 -- 172.21.15.133:0/4209115480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4108070 msgr2=0x7fcdb410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/4209115480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4108070 0x7fcdb410a4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b0a0 tx=0x7fcdac00b3b0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 -- 172.21.15.133:0/4209115480 shutdown_connections 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/4209115480 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4108070 0x7fcdb410a4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/4209115480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb4105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/4209115480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4105740 0x7fcdb4107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 -- 172.21.15.133:0/4209115480 >> 172.21.15.133:0/4209115480 conn(0x7fcdb40fc820 msgr2=0x7fcdb40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 -- 172.21.15.133:0/4209115480 shutdown_connections 2024-05-09T08:21:22.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.924+0000 7fcdb9a00700 1 -- 172.21.15.133:0/4209115480 wait complete. 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.925+0000 7fcdb9a00700 1 Processor -- start 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.925+0000 7fcdb9a00700 1 -- start start 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 0x7fcdb419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 0x7fcdb41a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb410dfa0 con 0x7fcdb4108070 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb410de20 con 0x7fcdb4105740 2024-05-09T08:21:22.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb9a00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb410e120 con 0x7fcdb4102e10 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb2a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdaa000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 0x7fcdb419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb2a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:38924/0 (socket says 172.21.15.133:38924) 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb2a00700 1 -- 172.21.15.133:0/2479669250 learned_addr learned my addr 172.21.15.133:0/2479669250 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdb3400700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 0x7fcdb41a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdaa000700 1 -- 172.21.15.133:0/2479669250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 msgr2=0x7fcdb419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdaa000700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.926+0000 7fcdaa000700 1 -- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 msgr2=0x7fcdb41a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdaa000700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 0x7fcdb41a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:22.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdaa000700 1 -- 172.21.15.133:0/2479669250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb41a2440 con 0x7fcdb4105740 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdb2a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdb3400700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 0x7fcdb41a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdaa000700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 0x7fcdb419ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00e9f0 tx=0x7fcd9c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00cbd0 con 0x7fcdb4105740 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd9c00cd70 con 0x7fcdb4105740 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0187a0 con 0x7fcdb4105740 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb41a2730 con 0x7fcdb4105740 2024-05-09T08:21:22.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.927+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb41a2c20 con 0x7fcdb4105740 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.928+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcd9c018940 con 0x7fcdb4105740 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.929+0000 7fcdabe00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 0x7fcd9803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.929+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd9c04c570 con 0x7fcdb4105740 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.929+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb4069010 con 0x7fcdb4105740 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.929+0000 7fcdb2a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 0x7fcd9803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:22.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.930+0000 7fcdb2a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 0x7fcd9803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcda4011590 tx=0x7fcda4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:22.934 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:22.654+0000 7f7743b32200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:21:22.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:22.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:22.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:22.935+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcd9c00fc70 con 0x7fcdb4105740 2024-05-09T08:21:23.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:23.204+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcdb4062760 con 0x7fcd98039330 2024-05-09T08:21:23.406 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:22.929+0000 7f73139ac200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:21:23.406 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:23.115+0000 7f73139ac200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:21:23.406 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:23.404+0000 7f73139ac200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:21:23.434 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.173+0000 7f7743b32200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:21:23.434 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.263+0000 7f7743b32200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:21:23.664 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:23.497+0000 7f73139ac200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:21:23.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:23.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T08:21:23.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:23.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:23.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:23.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:23.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.790 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:23.791 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:23.791 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.845 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.503+0000 7f7743b32200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.636+0000 7f7743b32200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.730+0000 7f7743b32200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:21:23.846 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:23.843+0000 7f7743b32200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:21:24.164 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:23.720+0000 7f73139ac200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:21:24.164 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:23.909+0000 7f73139ac200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:21:24.434 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:24.147+0000 7f7743b32200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:21:24.725 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:24 smithi133 ceph-mon[29610]: Reconfiguring daemon mgr.x on smithi133 2024-05-09T08:21:24.725 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:24 smithi133 ceph-mon[29610]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:24.725 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.725 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.914 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:24.530+0000 7f73139ac200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:21:24.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:24 smithi160 ceph-mon[34070]: Reconfiguring daemon mgr.x on smithi133 2024-05-09T08:21:24.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:24 smithi160 ceph-mon[34070]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:24.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:24 smithi188 ceph-mon[34645]: Reconfiguring daemon mgr.x on smithi133 2024-05-09T08:21:24.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:24 smithi188 ceph-mon[34645]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:24.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:24.934 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:24.744+0000 7f7743b32200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:21:25.252 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:24.956+0000 7f73139ac200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:21:25.252 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:25.051+0000 7f73139ac200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:21:25.252 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:25.251+0000 7f73139ac200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:21:25.434 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:21:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z[36376]: 2024-05-09T08:21:24.960+0000 7f7743b32200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:21:25.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:25.642+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7fcd9c0109a0 con 0x7fcdb4105740 2024-05-09T08:21:25.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:25.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:25.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:25.645 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: Standby manager daemon z started 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T08:21:25.646 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:25 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:25.878 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:25.784+0000 7f73139ac200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:21:25.878 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:25.876+0000 7f73139ac200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:21:25.878 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: Standby manager daemon z started 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T08:21:25.879 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:25 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:25.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:25.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:25.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: Standby manager daemon z started 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T08:21:25.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:25 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.188:0/3279648454' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:26.164 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:26.118+0000 7f73139ac200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:21:26.646 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:26.251+0000 7f73139ac200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:21:26.646 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:26.343+0000 7f73139ac200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:21:26.646 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:26.459+0000 7f73139ac200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:21:26.914 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:26.763+0000 7f73139ac200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:21:26.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:26 smithi160 ceph-mon[34070]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-09T08:21:26.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:26 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T08:21:26.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:26 smithi133 ceph-mon[29610]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-09T08:21:26.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:26 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T08:21:26.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:26 smithi188 ceph-mon[34645]: mgrmap e14: x(active, since 89s), standbys: z 2024-05-09T08:21:26.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:26 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T08:21:27.650 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:27 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:27.353+0000 7f73139ac200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:21:27.650 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:21:27 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y[35472]: 2024-05-09T08:21:27.571+0000 7f73139ac200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:21:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3399562406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]': finished 2024-05-09T08:21:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: Standby manager daemon y started 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T08:21:27.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:27 smithi160 ceph-mon[34070]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3399562406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]': finished 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: Standby manager daemon y started 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T08:21:27.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:27 smithi133 ceph-mon[29610]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3399562406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]: dispatch 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d560e467-02ea-4321-815b-434186255b78"}]': finished 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: Standby manager daemon y started 2024-05-09T08:21:27.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T08:21:27.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T08:21:27.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:27 smithi188 ceph-mon[34645]: from='mgr.? 172.21.15.160:0/26468625' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T08:21:28.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:28.540+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mon.2 v2:172.21.15.160:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fcd9c0173d0 con 0x7fcdb4105740 2024-05-09T08:21:28.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:28 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3315208277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:28.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:28 smithi160 ceph-mon[34070]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-09T08:21:28.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T08:21:28.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:28 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3315208277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:28.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:28 smithi133 ceph-mon[29610]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-09T08:21:28.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T08:21:28.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:28 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3315208277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:28.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:28 smithi188 ceph-mon[34645]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-05-09T08:21:28.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T08:21:29.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:29 smithi160 ceph-mon[34070]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:29 smithi133 ceph-mon[29610]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:29 smithi188 ceph-mon[34645]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:31 smithi160 ceph-mon[34070]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:31.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:31 smithi133 ceph-mon[29610]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:31.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:31 smithi188 ceph-mon[34645]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:32.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:32 smithi133 ceph-mon[29610]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:33.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:32 smithi160 ceph-mon[34070]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:33.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:32 smithi188 ceph-mon[34645]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:35.892 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:35 smithi133 ceph-mon[29610]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:35.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:35 smithi160 ceph-mon[34070]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:35.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:35 smithi188 ceph-mon[34645]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:37.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:37 smithi160 ceph-mon[34070]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:37.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T08:21:37.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:37.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:37 smithi133 ceph-mon[29610]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:37.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T08:21:37.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:37.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:37 smithi188 ceph-mon[34645]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:37.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T08:21:37.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:38.670 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:38 smithi133 ceph-mon[29610]: Deploying daemon osd.0 on smithi133 2024-05-09T08:21:38.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:38 smithi160 ceph-mon[34070]: Deploying daemon osd.0 on smithi133 2024-05-09T08:21:38.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:38 smithi188 ceph-mon[34645]: Deploying daemon osd.0 on smithi133 2024-05-09T08:21:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:39 smithi133 ceph-mon[29610]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:39.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:39 smithi160 ceph-mon[34070]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:39.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:39 smithi188 ceph-mon[34645]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:41.775 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:41 smithi133 ceph-mon[29610]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:41.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:41 smithi160 ceph-mon[34070]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:41.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:41 smithi188 ceph-mon[34645]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:43.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:43 smithi160 ceph-mon[34070]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:43.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:43 smithi133 ceph-mon[29610]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:43.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:43 smithi188 ceph-mon[34645]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:45.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:45.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:45 smithi133 ceph-mon[29610]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:45.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:45.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:45 smithi160 ceph-mon[34070]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:45.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:45.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:45.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:45 smithi188 ceph-mon[34645]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:47.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:47.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:47.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:48.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:48 smithi160 ceph-mon[34070]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:48.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:48 smithi160 ceph-mon[34070]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:48.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:48 smithi133 ceph-mon[29610]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:48.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:48 smithi133 ceph-mon[29610]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:48.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:48 smithi188 ceph-mon[34645]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:48.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:48 smithi188 ceph-mon[34645]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T08:21:49.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.195+0000 7fcdabe00700 1 -- 172.21.15.133:0/2479669250 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcdb4062760 con 0x7fcd98039330 2024-05-09T08:21:49.198 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 0 on host 'smithi133' 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 msgr2=0x7fcd9803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 0x7fcd9803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcda4011590 tx=0x7fcda4009260 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 msgr2=0x7fcdb419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 0x7fcdb419ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00e9f0 tx=0x7fcd9c00ed00 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 shutdown_connections 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb4108070 0x7fcdb41a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcd98039330 0x7fcd9803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb4102e10 0x7fcdb419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 --2- 172.21.15.133:0/2479669250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb4105740 0x7fcdb419ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:49.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.200+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 >> 172.21.15.133:0/2479669250 conn(0x7fcdb40fc820 msgr2=0x7fcdb4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.201+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 shutdown_connections 2024-05-09T08:21:49.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:21:49.201+0000 7fcdb9a00700 1 -- 172.21.15.133:0/2479669250 wait complete. 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:49.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:49.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:21:50.005 DEBUG:teuthology.orchestra.run.smithi133:osd.0> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.0.service 2024-05-09T08:21:50.010 INFO:tasks.cephadm:Deploying osd.1 on smithi160 with /dev/vg_nvme/lv_4... 2024-05-09T08:21:50.010 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T08:21:50.247 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:21:50.580 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:21:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0[44303]: 2024-05-09T08:21:50.288+0000 7f3391600700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T08:21:50.580 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:21:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0[44303]: 2024-05-09T08:21:50.312+0000 7f3387400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:21:50.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:50 smithi160 ceph-mon[34070]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:21:50.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:50 smithi160 ceph-mon[34070]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T08:21:50.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:50 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:50 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:50 smithi160 ceph-mon[34070]: from='osd.0 ' entity='osd.0' 2024-05-09T08:21:50.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:50 smithi133 ceph-mon[29610]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:21:50.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:50 smithi133 ceph-mon[29610]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T08:21:50.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:50 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:50 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:50 smithi133 ceph-mon[29610]: from='osd.0 ' entity='osd.0' 2024-05-09T08:21:50.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:50 smithi188 ceph-mon[34645]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:21:50.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:50 smithi188 ceph-mon[34645]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T08:21:50.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:50 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:50 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:50.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:50 smithi188 ceph-mon[34645]: from='osd.0 ' entity='osd.0' 2024-05-09T08:21:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:21:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:21:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:51.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] boot 2024-05-09T08:21:51.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T08:21:51.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] boot 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T08:21:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: osd.0 [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] boot 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T08:21:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T08:21:52.109 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T08:21:52.110 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T08:21:52.110 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-09T08:21:52.110 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-09T08:21:52.110 INFO:teuthology.orchestra.run.smithi160.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113358 s, 925 MB/s 2024-05-09T08:21:52.110 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-09T08:21:52.725 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2024-05-09T08:21:52.959 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:21:53.374 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:53 smithi160 ceph-mon[34070]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T08:21:53.374 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:53 smithi160 ceph-mon[34070]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:53.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:53 smithi133 ceph-mon[29610]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T08:21:53.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:53 smithi133 ceph-mon[29610]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:53.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:53 smithi188 ceph-mon[34645]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T08:21:53.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:53 smithi188 ceph-mon[34645]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:54.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 -- 172.21.15.160:0/598379451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 msgr2=0x7f7324105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 --2- 172.21.15.160:0/598379451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324105290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7314009990 tx=0x7f7314009ca0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 -- 172.21.15.160:0/598379451 shutdown_connections 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 --2- 172.21.15.160:0/598379451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324108100 0x7f732410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 --2- 172.21.15.160:0/598379451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324105290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 --2- 172.21.15.160:0/598379451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73241057d0 0x7f7324107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 -- 172.21.15.160:0/598379451 >> 172.21.15.160:0/598379451 conn(0x7f73240fc810 msgr2=0x7f73240fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 -- 172.21.15.160:0/598379451 shutdown_connections 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.331+0000 7f732c5f4700 1 -- 172.21.15.160:0/598379451 wait complete. 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 Processor -- start 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 -- start start 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:54.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 0x7f7324198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7324108100 0x7f732419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f732410e090 con 0x7f7324108100 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f732410df10 con 0x7f7324102ea0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732c5f4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f732410e210 con 0x7f73241057d0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43472/0 (socket says 172.21.15.160:43472) 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 -- 172.21.15.160:0/2913129127 learned_addr learned my addr 172.21.15.160:0/2913129127 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f7329b8f700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 0x7f7324198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 -- 172.21.15.160:0/2913129127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 msgr2=0x7f7324198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 0x7f7324198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 -- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7324108100 msgr2=0x7f732419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7324108100 0x7f732419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f732a390700 1 -- 172.21.15.160:0/2913129127 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73241a0360 con 0x7f7324102ea0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.332+0000 7f7329b8f700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 0x7f7324198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f732a390700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7314003890 tx=0x7f7314003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:54.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314027070 con 0x7f7324102ea0 2024-05-09T08:21:54.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314022470 con 0x7f7324102ea0 2024-05-09T08:21:54.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73241a05f0 con 0x7f7324102ea0 2024-05-09T08:21:54.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400f6a0 con 0x7f7324102ea0 2024-05-09T08:21:54.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.333+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a0ac0 con 0x7f7324102ea0 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.334+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7324069010 con 0x7f7324102ea0 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.335+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f731402a070 con 0x7f7324102ea0 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.335+0000 7f731b7fe700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 0x7f73100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.335+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f73140ce6a0 con 0x7f7324102ea0 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.336+0000 7f7329b8f700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 0x7f73100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:21:54.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.337+0000 7f7329b8f700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 0x7f73100a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73200099d0 tx=0x7f7320008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:21:54.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.340+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f731405e740 con 0x7f7324102ea0 2024-05-09T08:21:54.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:21:54.549+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7324062760 con 0x7f73100a2b90 2024-05-09T08:21:55.810 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi133 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 17089M 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:55.811 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:55.812 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi133 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 17089M 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:55.931 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi133 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 17089M 2024-05-09T08:21:55.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:21:55.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:21:57.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:57.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/836781061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]': finished 2024-05-09T08:21:57.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T08:21:57.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/836781061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]': finished 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T08:21:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/836781061' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]: dispatch 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "082e4943-4828-4bdd-a221-337d3c05a7ee"}]': finished 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T08:21:57.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:21:58.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:58 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/4270151383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:58 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/4270151383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:58.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:58 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/4270151383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:21:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:21:59 smithi160 ceph-mon[34070]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:59.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:21:59 smithi133 ceph-mon[29610]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:21:59.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:21:59 smithi188 ceph-mon[34645]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:01.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:01 smithi160 ceph-mon[34070]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:01.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:01 smithi133 ceph-mon[29610]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:01.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:01 smithi188 ceph-mon[34645]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:03.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:03 smithi160 ceph-mon[34070]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:03.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:03 smithi133 ceph-mon[29610]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:03.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:03 smithi188 ceph-mon[34645]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:04.899 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T08:22:04.899 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:04.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T08:22:04.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:04.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T08:22:04.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:05.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:05 smithi160 ceph-mon[34070]: Deploying daemon osd.1 on smithi160 2024-05-09T08:22:05.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:05 smithi160 ceph-mon[34070]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:05.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:05 smithi133 ceph-mon[29610]: Deploying daemon osd.1 on smithi160 2024-05-09T08:22:05.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:05 smithi133 ceph-mon[29610]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:05 smithi188 ceph-mon[34645]: Deploying daemon osd.1 on smithi160 2024-05-09T08:22:05.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:05 smithi188 ceph-mon[34645]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:07.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:07 smithi160 ceph-mon[34070]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:07.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:07 smithi133 ceph-mon[29610]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:07.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:07 smithi188 ceph-mon[34645]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:09 smithi160 ceph-mon[34070]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:09 smithi133 ceph-mon[29610]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:09 smithi188 ceph-mon[34645]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:11.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:11 smithi160 ceph-mon[34070]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:11 smithi133 ceph-mon[29610]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:11.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:11 smithi188 ceph-mon[34645]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.840+0000 7f731b7fe700 1 -- 172.21.15.160:0/2913129127 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7324062760 con 0x7f73100a2b90 2024-05-09T08:22:12.842 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 1 on host 'smithi160' 2024-05-09T08:22:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 msgr2=0x7f73100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 0x7f73100a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73200099d0 tx=0x7f7320008040 comp rx=0 tx=0).stop 2024-05-09T08:22:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 msgr2=0x7f7324198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7314003890 tx=0x7f7314003730 comp rx=0 tx=0).stop 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 shutdown_connections 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7324108100 0x7f732419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7324102ea0 0x7f7324198700 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f73100a2b90 0x7f73100a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 --2- 172.21.15.160:0/2913129127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73241057d0 0x7f7324198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 >> 172.21.15.160:0/2913129127 conn(0x7f73240fc810 msgr2=0x7f7324103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 shutdown_connections 2024-05-09T08:22:12.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:22:12.843+0000 7f732c5f4700 1 -- 172.21.15.160:0/2913129127 wait complete. 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:13.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:13.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:13.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:13.474 DEBUG:teuthology.orchestra.run.smithi160:osd.1> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.1.service 2024-05-09T08:22:13.477 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-05-09T08:22:13.477 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T08:22:13.720 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:22:14.120 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:14 smithi188 ceph-mon[34645]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:14.120 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:14 smithi188 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:14.361 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:14 smithi160 ceph-mon[34070]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:14.362 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:14 smithi160 ceph-mon[34070]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:14.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:14 smithi133 ceph-mon[29610]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:14.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:14 smithi133 ceph-mon[29610]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:15.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:15 smithi160 ceph-mon[34070]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:15.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:15 smithi133 ceph-mon[29610]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: from='osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:15.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:15 smithi188 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113225 s, 926 MB/s 2024-05-09T08:22:15.546 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-09T08:22:15.786 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1[38657]: 2024-05-09T08:22:15.513+0000 7fec1351b700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T08:22:15.786 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1[38657]: 2024-05-09T08:22:15.516+0000 7fec0b306700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:22:16.105 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:16 smithi188 ceph-mon[34645]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T08:22:16.127 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.127 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.127 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:16 smithi188 ceph-mon[34645]: from='osd.1 ' entity='osd.1' 2024-05-09T08:22:16.127 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-05-09T08:22:16.367 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:22:16.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:16 smithi160 ceph-mon[34070]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T08:22:16.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:16 smithi160 ceph-mon[34070]: from='osd.1 ' entity='osd.1' 2024-05-09T08:22:16.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:16 smithi133 ceph-mon[29610]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T08:22:16.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:16.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:16 smithi133 ceph-mon[29610]: from='osd.1 ' entity='osd.1' 2024-05-09T08:22:16.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi160 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:17.332 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] boot 2024-05-09T08:22:17.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T08:22:17.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:17.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:17.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:17 smithi133 ceph-mon[29610]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi160 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:17.350 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] boot 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.351 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:17 smithi188 ceph-mon[34645]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:17.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi160 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 17088M 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: osd.1 [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] boot 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T08:22:17.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:17.416 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:17.416 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:17 smithi160 ceph-mon[34070]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-09T08:22:17.960 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 -- 172.21.15.188:0/2795209341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c102ed0 msgr2=0x7f866c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:17.960 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 --2- 172.21.15.188:0/2795209341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c102ed0 0x7f866c1052c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f865c009990 tx=0x7f865c009ca0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 -- 172.21.15.188:0/2795209341 shutdown_connections 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 --2- 172.21.15.188:0/2795209341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f866c108130 0x7f866c10a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 --2- 172.21.15.188:0/2795209341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c102ed0 0x7f866c1052c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 --2- 172.21.15.188:0/2795209341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c105800 0x7f866c107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 -- 172.21.15.188:0/2795209341 >> 172.21.15.188:0/2795209341 conn(0x7f866c0fc820 msgr2=0x7f866c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 -- 172.21.15.188:0/2795209341 shutdown_connections 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.957+0000 7f867213c700 1 -- 172.21.15.188:0/2795209341 wait complete. 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 Processor -- start 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 -- start start 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f866c105800 0x7f866c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c108130 0x7f866c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:17.961 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10dfa0 con 0x7f866c105800 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10de20 con 0x7f866c108130 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f867213c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f866c10e120 con 0x7f866c102ed0 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:36036/0 (socket says 172.21.15.188:36036) 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 -- 172.21.15.188:0/424473461 learned_addr learned my addr 172.21.15.188:0/424473461 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866bfff700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c108130 0x7f866c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 -- 172.21.15.188:0/424473461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c108130 msgr2=0x7f866c19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c108130 0x7f866c19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 -- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f866c105800 msgr2=0x7f866c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f866c105800 0x7f866c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.958+0000 7f866b7fe700 1 -- 172.21.15.188:0/424473461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866c1a0220 con 0x7f866c102ed0 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f866b7fe700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f865c005f50 tx=0x7f865c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:22:17.962 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c026070 con 0x7f866c102ed0 2024-05-09T08:22:17.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866c1a04b0 con 0x7f866c102ed0 2024-05-09T08:22:17.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865c00f460 con 0x7f866c102ed0 2024-05-09T08:22:17.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c021670 con 0x7f866c102ed0 2024-05-09T08:22:17.963 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.959+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866c1a0960 con 0x7f866c102ed0 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.960+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f866c069010 con 0x7f866c102ed0 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.961+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f865c029070 con 0x7f866c102ed0 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.961+0000 7f8668ff9700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 0x7f86540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.961+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f865c0ce7a0 con 0x7f866c102ed0 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.962+0000 7f866affd700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 0x7f86540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:17.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.963+0000 7f866affd700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 0x7f86540a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8660005d90 tx=0x7f8660005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:22:17.967 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:17.966+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f865c05d740 con 0x7f866c102ed0 2024-05-09T08:22:18.172 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:18.171+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f866c062760 con 0x7f86540a2b90 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:19.363 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:19 smithi188 ceph-mon[34645]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:19.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T08:22:19.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:19.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:19.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:19.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:19.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:19 smithi160 ceph-mon[34070]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:19.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:19 smithi133 ceph-mon[29610]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:21.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:21.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/2813411309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]': finished 2024-05-09T08:22:21.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T08:22:21.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:21.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:21 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3887505601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/2813411309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]': finished 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:21.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:21 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3887505601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/2813411309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]: dispatch 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "228eae91-e83f-4241-ae1f-a4f38890d2db"}]': finished 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:21.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:21 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3887505601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:23.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:23 smithi160 ceph-mon[34070]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:23.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:23 smithi133 ceph-mon[29610]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:23.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:23 smithi188 ceph-mon[34645]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:25.912 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:25 smithi188 ceph-mon[34645]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:25.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:25 smithi160 ceph-mon[34070]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:25.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:25 smithi133 ceph-mon[29610]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:27.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:27 smithi188 ceph-mon[34645]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:27 smithi160 ceph-mon[34070]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:27 smithi133 ceph-mon[29610]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:28.902 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T08:22:28.902 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:28.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T08:22:28.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:28.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T08:22:28.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:29.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:29 smithi188 ceph-mon[34645]: Deploying daemon osd.2 on smithi188 2024-05-09T08:22:29.862 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:29 smithi188 ceph-mon[34645]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:29.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:29 smithi160 ceph-mon[34070]: Deploying daemon osd.2 on smithi188 2024-05-09T08:22:29.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:29 smithi160 ceph-mon[34070]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:29 smithi133 ceph-mon[29610]: Deploying daemon osd.2 on smithi188 2024-05-09T08:22:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:29 smithi133 ceph-mon[29610]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:31 smithi160 ceph-mon[34070]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:31 smithi133 ceph-mon[29610]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:31.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:31 smithi188 ceph-mon[34645]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:33.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:33 smithi160 ceph-mon[34070]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:33.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:33 smithi133 ceph-mon[29610]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:33.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:33 smithi188 ceph-mon[34645]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:33.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:33.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:35.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:35 smithi160 ceph-mon[34070]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:35.917 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:35 smithi188 ceph-mon[34645]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:35 smithi133 ceph-mon[29610]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:36.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:36 smithi188 ceph-mon[34645]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:37.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.153+0000 7f8668ff9700 1 -- 172.21.15.188:0/424473461 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f866c062760 con 0x7f86540a2b90 2024-05-09T08:22:37.155 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-05-09T08:22:37.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 msgr2=0x7f86540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:37.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 0x7f86540a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8660005d90 tx=0x7f8660005ce0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 msgr2=0x7f866c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:37.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f865c005f50 tx=0x7f865c00bbb0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 shutdown_connections 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f866c108130 0x7f866c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f86540a2b90 0x7f86540a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f866c102ed0 0x7f866c1985c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 --2- 172.21.15.188:0/424473461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f866c105800 0x7f866c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 >> 172.21.15.188:0/424473461 conn(0x7f866c0fc820 msgr2=0x7f866c103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 shutdown_connections 2024-05-09T08:22:37.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:22:37.156+0000 7f867213c700 1 -- 172.21.15.188:0/424473461 wait complete. 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:36 smithi160 ceph-mon[34070]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:36 smithi133 ceph-mon[29610]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:37.868 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.2.service 2024-05-09T08:22:37.871 INFO:tasks.cephadm:Deploying osd.3 on smithi133 with /dev/vg_nvme/lv_3... 2024-05-09T08:22:37.871 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T08:22:38.299 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:22:38.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:38.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:38 smithi188 ceph-mon[34645]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:38.411 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:38 smithi188 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:38 smithi160 ceph-mon[34070]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:38 smithi160 ceph-mon[34070]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:38.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:38.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:22:38.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:38 smithi133 ceph-mon[29610]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:38.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:38 smithi133 ceph-mon[29610]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:39 smithi160 ceph-mon[34070]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:39 smithi133 ceph-mon[29610]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: from='osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:22:39.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:39 smithi188 ceph-mon[34645]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:39.934 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2[39647]: 2024-05-09T08:22:39.590+0000 7ff5ed19b700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T08:22:39.934 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2[39647]: 2024-05-09T08:22:39.593+0000 7ff5e4f86700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:22:40.405 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:40 smithi188 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:22:40.405 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:40 smithi188 ceph-mon[34645]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T08:22:40.406 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.406 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.406 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:40 smithi188 ceph-mon[34645]: from='osd.2 ' entity='osd.2' 2024-05-09T08:22:40.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:40 smithi160 ceph-mon[34070]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:22:40.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:40 smithi160 ceph-mon[34070]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T08:22:40.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:40 smithi160 ceph-mon[34070]: from='osd.2 ' entity='osd.2' 2024-05-09T08:22:40.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:40 smithi133 ceph-mon[29610]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:22:40.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:40 smithi133 ceph-mon[29610]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T08:22:40.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:40.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:40 smithi133 ceph-mon[29610]: from='osd.2 ' entity='osd.2' 2024-05-09T08:22:41.008 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T08:22:41.008 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T08:22:41.008 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-05-09T08:22:41.008 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-05-09T08:22:41.009 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0350706 s, 299 MB/s 2024-05-09T08:22:41.009 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-05-09T08:22:41.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] boot 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] boot 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: osd.2 [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] boot 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T08:22:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:41.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-05-09T08:22:42.191 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:22:42.502 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi188 2024-05-09T08:22:42.502 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.502 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.502 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T08:22:42.503 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi188 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:42.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T08:22:42.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi188 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 17088M 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T08:22:42.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:22:43.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:43 smithi133 ceph-mon[29610]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:43.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:43.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:43 smithi133 ceph-mon[29610]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T08:22:43.852 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:43 smithi133 ceph-mon[29610]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:22:43.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:43 smithi160 ceph-mon[34070]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:43.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:43.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:43 smithi160 ceph-mon[34070]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T08:22:43.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:43 smithi160 ceph-mon[34070]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:22:43.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:43 smithi188 ceph-mon[34645]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:43.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:22:43.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:43 smithi188 ceph-mon[34645]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T08:22:43.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:43 smithi188 ceph-mon[34645]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:22:44.123 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:43 smithi133 sudo[48930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:44.123 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:43 smithi133 sudo[48930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:44.123 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:43 smithi133 sudo[48930]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:44.123 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:43 smithi133 sudo[48930]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.381+0000 7fb696000700 1 -- 172.21.15.133:0/171522387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 msgr2=0x7fb690075200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.381+0000 7fb696000700 1 --2- 172.21.15.133:0/171522387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb690075200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c009ca0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 -- 172.21.15.133:0/171522387 shutdown_connections 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 --2- 172.21.15.133:0/171522387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69010a260 0x7fb69010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 --2- 172.21.15.133:0/171522387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb690076350 0x7fb6900747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 --2- 172.21.15.133:0/171522387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb690075200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 -- 172.21.15.133:0/171522387 >> 172.21.15.133:0/171522387 conn(0x7fb6900fd930 msgr2=0x7fb6900ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 -- 172.21.15.133:0/171522387 shutdown_connections 2024-05-09T08:22:44.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.382+0000 7fb696000700 1 -- 172.21.15.133:0/171522387 wait complete. 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.383+0000 7fb696000700 1 Processor -- start 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 -- start start 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690076350 0x7fb69019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 0x7fb6901a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901101c0 con 0x7fb690074d80 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110040 con 0x7fb690076350 2024-05-09T08:22:44.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb696000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb690110340 con 0x7fb69010a260 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb68f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb68f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57392/0 (socket says 172.21.15.133:57392) 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb68f400700 1 -- 172.21.15.133:0/1534955972 learned_addr learned my addr 172.21.15.133:0/1534955972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.384+0000 7fb68ea00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690076350 0x7fb69019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68fe00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 0x7fb6901a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 -- 172.21.15.133:0/1534955972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 msgr2=0x7fb6901a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 0x7fb6901a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 -- 172.21.15.133:0/1534955972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690076350 msgr2=0x7fb69019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690076350 0x7fb69019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 -- 172.21.15.133:0/1534955972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6901a2460 con 0x7fb690074d80 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68fe00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 0x7fb6901a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:22:44.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb68f400700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb68000e9a0 tx=0x7fb68000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68000cbd0 con 0x7fb690074d80 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb68000cd70 con 0x7fb690074d80 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.385+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6901a2750 con 0x7fb690074d80 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.386+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680010430 con 0x7fb690074d80 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.386+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6901a2ce0 con 0x7fb690074d80 2024-05-09T08:22:44.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.387+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb690069010 con 0x7fb690074d80 2024-05-09T08:22:44.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.389+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fb680004750 con 0x7fb690074d80 2024-05-09T08:22:44.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.390+0000 7fb687e00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 0x7fb6740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:22:44.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.391+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fb6800ce080 con 0x7fb690074d80 2024-05-09T08:22:44.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.391+0000 7fb68ea00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 0x7fb6740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:22:44.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.391+0000 7fb68ea00700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 0x7fb6740a50d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000c00 tx=0x7fb67c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:22:44.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.398+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb68009c480 con 0x7fb690074d80 2024-05-09T08:22:44.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:44.634+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb690062760 con 0x7fb6740a2c10 2024-05-09T08:22:44.637 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:44 smithi133 ceph-mon[29610]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T08:22:44.637 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:44.638 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:44 smithi133 sudo[48959]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:44.638 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:44 smithi133 sudo[48959]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:44.638 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:44 smithi133 sudo[48959]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:44.638 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:22:44 smithi133 sudo[48959]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:44.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:44 smithi160 ceph-mon[34070]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T08:22:44.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:44.914 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:44 smithi160 sudo[42657]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:44.914 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:44 smithi160 sudo[42657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:44.914 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:44 smithi160 sudo[42657]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:44.914 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:44 smithi160 sudo[42657]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:44.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:44 smithi188 ceph-mon[34645]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T08:22:44.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:45.414 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:45 smithi160 sudo[42660]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:45.414 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:45 smithi160 sudo[42660]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:45.414 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:45 smithi160 sudo[42660]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:45.414 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:22:45 smithi160 sudo[42660]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:45.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:22:45.613+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb6800983f0 con 0x7fb690074d80 2024-05-09T08:22:45.629 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:45.629 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:45.630 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43719]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:45.630 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43719]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: Cluster is now healthy 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:45.838 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:45 smithi133 ceph-mon[29610]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: Cluster is now healthy 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:45.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:45.915 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:45 smithi160 ceph-mon[34070]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T08:22:45.933 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:45.934 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:45.934 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:45.934 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:22:45 smithi188 sudo[43722]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: Cluster is now healthy 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:22:45.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:22:45.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:45.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:45.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:45 smithi188 ceph-mon[34645]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T08:22:46.576 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:46.576 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:46.576 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:46.576 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49100]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:46.826 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49108]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:46.826 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:46.826 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49108]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:46.827 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 sudo[49108]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:47.089 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:46 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:47.089 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:46 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:47.089 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:47.089 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:47.089 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:47.090 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43725]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:47.130 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:47.255 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:46 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:47.255 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:46 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:22:47.255 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:46 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:47.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:47.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:47.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43728]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:47.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 sudo[43728]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:47.830 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T08:22:47.831 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:47.831 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42663]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:47.831 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42663]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42666]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 sudo[42666]: pam_unix(sudo:session): session closed for user root 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:47 smithi160 ceph-mon[34070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.181 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:47 smithi133 ceph-mon[29610]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T08:22:48.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T08:22:48.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T08:22:48.185 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:47 smithi188 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T08:22:49.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:48 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]: dispatch 2024-05-09T08:22:49.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:48 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]': finished 2024-05-09T08:22:49.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:48 smithi160 ceph-mon[34070]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T08:22:49.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:48 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:22:49.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:48 smithi160 ceph-mon[34070]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:49.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:48 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]: dispatch 2024-05-09T08:22:49.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:48 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]': finished 2024-05-09T08:22:49.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:48 smithi133 ceph-mon[29610]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T08:22:49.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:48 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:22:49.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:48 smithi133 ceph-mon[29610]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:49.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:48 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]: dispatch 2024-05-09T08:22:49.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:48 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2282474267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c20bfe2a-6caf-485b-b3d2-4d661db1ffef"}]': finished 2024-05-09T08:22:49.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:48 smithi188 ceph-mon[34645]: osdmap e22: 4 total, 3 up, 4 in 2024-05-09T08:22:49.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:48 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:22:49.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:48 smithi188 ceph-mon[34645]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:49 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2582438988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:50.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:49 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2582438988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:49 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2582438988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:22:51.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:50 smithi160 ceph-mon[34070]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:51.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:50 smithi133 ceph-mon[29610]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:51.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:50 smithi188 ceph-mon[34645]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:22:53.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:53 smithi133 ceph-mon[29610]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 64 KiB/s wr, 5 op/s 2024-05-09T08:22:53.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:53 smithi160 ceph-mon[34070]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 64 KiB/s wr, 5 op/s 2024-05-09T08:22:53.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:53 smithi188 ceph-mon[34645]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 64 KiB/s wr, 5 op/s 2024-05-09T08:22:55.881 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:55 smithi133 ceph-mon[29610]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 43 KiB/s wr, 3 op/s 2024-05-09T08:22:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:55 smithi160 ceph-mon[34070]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 43 KiB/s wr, 3 op/s 2024-05-09T08:22:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:55 smithi188 ceph-mon[34645]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 43 KiB/s wr, 3 op/s 2024-05-09T08:22:57.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:57 smithi160 ceph-mon[34070]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-05-09T08:22:57.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T08:22:57.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:57 smithi133 ceph-mon[29610]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-05-09T08:22:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T08:22:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:57 smithi188 ceph-mon[34645]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-05-09T08:22:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T08:22:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:22:58.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:58 smithi133 ceph-mon[29610]: Deploying daemon osd.3 on smithi133 2024-05-09T08:22:58.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:58 smithi160 ceph-mon[34070]: Deploying daemon osd.3 on smithi133 2024-05-09T08:22:58.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:58 smithi188 ceph-mon[34645]: Deploying daemon osd.3 on smithi133 2024-05-09T08:22:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:22:59 smithi160 ceph-mon[34070]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 912 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T08:22:59.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:22:59 smithi133 ceph-mon[29610]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 912 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T08:22:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:22:59 smithi188 ceph-mon[34645]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 912 B/s rd, 26 KiB/s wr, 2 op/s 2024-05-09T08:23:01.866 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:01 smithi133 ceph-mon[29610]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:01.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:01 smithi160 ceph-mon[34070]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:01.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:01 smithi188 ceph-mon[34645]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:03.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:03 smithi160 ceph-mon[34070]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:03.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:03 smithi133 ceph-mon[29610]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:03.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:03 smithi188 ceph-mon[34645]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-05-09T08:23:05.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:05 smithi160 ceph-mon[34070]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:05.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:05 smithi133 ceph-mon[29610]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:05 smithi188 ceph-mon[34645]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:07.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:07.076 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:06 smithi133 ceph-mon[29610]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:07.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:07.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:06 smithi160 ceph-mon[34070]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:07.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:07.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:07.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:06 smithi188 ceph-mon[34645]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:09.887 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:09.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:10.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.677+0000 7fb687e00700 1 -- 172.21.15.133:0/1534955972 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb690062760 con 0x7fb6740a2c10 2024-05-09T08:23:10.679 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 3 on host 'smithi133' 2024-05-09T08:23:10.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 msgr2=0x7fb6740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:10.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 0x7fb6740a50d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000c00 tx=0x7fb67c005fb0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 msgr2=0x7fb69019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:10.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb68000e9a0 tx=0x7fb68000ecb0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 shutdown_connections 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6740a2c10 0x7fb6740a50d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb69010a260 0x7fb6901a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb690074d80 0x7fb69019a800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 --2- 172.21.15.133:0/1534955972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb690076350 0x7fb69019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 >> 172.21.15.133:0/1534955972 conn(0x7fb6900fd930 msgr2=0x7fb690106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 shutdown_connections 2024-05-09T08:23:10.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:23:10.681+0000 7fb696000700 1 -- 172.21.15.133:0/1534955972 wait complete. 2024-05-09T08:23:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:10 smithi160 ceph-mon[34070]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T08:23:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:10 smithi133 ceph-mon[29610]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T08:23:10.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:10 smithi188 ceph-mon[34645]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T08:23:11.412 DEBUG:teuthology.orchestra.run.smithi133:osd.3> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.3.service 2024-05-09T08:23:11.417 INFO:tasks.cephadm:Deploying osd.4 on smithi160 with /dev/vg_nvme/lv_3... 2024-05-09T08:23:11.417 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T08:23:11.663 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:11.783 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:11 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:11.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:11 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:11.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:11.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:11.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:11 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:12.656 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:23:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:23:12.364+0000 7f2911a00700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T08:23:12.657 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:23:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:23:12.393+0000 7f2906e00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:23:12.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:23:12.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T08:23:12.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T08:23:12.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.912 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:12 smithi133 ceph-mon[29610]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' 2024-05-09T08:23:12.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:12 smithi160 ceph-mon[34070]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' 2024-05-09T08:23:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:12.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:12 smithi188 ceph-mon[34645]: from='osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022]' entity='osd.3' 2024-05-09T08:23:13.559 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T08:23:13.560 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T08:23:13.560 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-09T08:23:13.560 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-09T08:23:13.560 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011491 s, 913 MB/s 2024-05-09T08:23:13.560 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] boot 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T08:23:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:23:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:23:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:13.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:13.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] boot 2024-05-09T08:23:13.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T08:23:13.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: osd.3 [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] boot 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: osdmap e26: 4 total, 4 up, 4 in 2024-05-09T08:23:13.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T08:23:14.256 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2024-05-09T08:23:14.511 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:23:15.166 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:14 smithi160 ceph-mon[34070]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T08:23:15.166 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:14 smithi160 ceph-mon[34070]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:15.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:14 smithi133 ceph-mon[29610]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T08:23:15.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:14 smithi133 ceph-mon[29610]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:15.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:14 smithi188 ceph-mon[34645]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T08:23:15.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:14 smithi188 ceph-mon[34645]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 -- 172.21.15.160:0/3175408728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198103610 msgr2=0x7f2198103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 --2- 172.21.15.160:0/3175408728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198103610 0x7f2198103a70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2188009990 tx=0x7f2188009ca0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 -- 172.21.15.160:0/3175408728 shutdown_connections 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 --2- 172.21.15.160:0/3175408728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103fb0 0x7f219810a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 --2- 172.21.15.160:0/3175408728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198100630 0x7f2198100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 --2- 172.21.15.160:0/3175408728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198103610 0x7f2198103a70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 -- 172.21.15.160:0/3175408728 >> 172.21.15.160:0/3175408728 conn(0x7f21980fbfb0 msgr2=0x7f21980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 -- 172.21.15.160:0/3175408728 shutdown_connections 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.931+0000 7f219fecf700 1 -- 172.21.15.160:0/3175408728 wait complete. 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 Processor -- start 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 -- start start 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103610 0x7f2198198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198103fb0 0x7f219819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f219810e040 con 0x7f2198103610 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f219810dec0 con 0x7f2198100630 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219fecf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f219810e1c0 con 0x7f2198103fb0 2024-05-09T08:23:15.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34050/0 (socket says 172.21.15.160:34050) 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 -- 172.21.15.160:0/1459304728 learned_addr learned my addr 172.21.15.160:0/1459304728 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219d46a700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103610 0x7f2198198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219e46c700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198103fb0 0x7f219819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 -- 172.21.15.160:0/1459304728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198103fb0 msgr2=0x7f219819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198103fb0 0x7f219819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 -- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103610 msgr2=0x7f2198198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103610 0x7f2198198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 -- 172.21.15.160:0/1459304728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21981a0340 con 0x7f2198100630 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.932+0000 7f219dc6b700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f219400b9c0 tx=0x7f219400bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.933+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194019070 con 0x7f2198100630 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.933+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2194014440 con 0x7f2198100630 2024-05-09T08:23:15.936 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.933+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21981a0630 con 0x7f2198100630 2024-05-09T08:23:15.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.933+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940125a0 con 0x7f2198100630 2024-05-09T08:23:15.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.933+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21981a0bf0 con 0x7f2198100630 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.934+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2198069010 con 0x7f2198100630 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.935+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f219401c070 con 0x7f2198100630 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.935+0000 7f218effd700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 0x7f21840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.936+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f21940cd280 con 0x7f2198100630 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.936+0000 7f219d46a700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 0x7f21840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:15.938 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.937+0000 7f219d46a700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 0x7f21840a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f218800b5c0 tx=0x7f2188005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:23:15.941 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:15.940+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f219409b400 con 0x7f2198100630 2024-05-09T08:23:16.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:16.145+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2198062760 con 0x7f21840a2c20 2024-05-09T08:23:16.146 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:15 smithi160 ceph-mon[34070]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T08:23:16.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:15 smithi133 ceph-mon[29610]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T08:23:16.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:15 smithi188 ceph-mon[34645]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T08:23:17.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T08:23:17.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:17.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:17.098 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:17.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:17.099 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:16 smithi160 ceph-mon[34070]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:17.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:16 smithi133 ceph-mon[29610]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:17.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T08:23:17.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:17.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:17.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:17.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:17.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:16 smithi188 ceph-mon[34645]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi133 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.005 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 8544M 2024-05-09T08:23:18.006 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.006 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:18.006 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:18.006 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi133 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 8544M 2024-05-09T08:23:18.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:18.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:18.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: osdmap e30: 4 total, 4 up, 4 in 2024-05-09T08:23:18.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi133 2024-05-09T08:23:18.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 8544M 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:18.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/93615573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]': finished 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T08:23:19.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/93615573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]': finished 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T08:23:19.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:19.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:23:19.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/93615573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]: dispatch 2024-05-09T08:23:19.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32"}]': finished 2024-05-09T08:23:19.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T08:23:19.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:19 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/283207561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:19 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/283207561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:20.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:19 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/283207561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:20 smithi160 ceph-mon[34070]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:23:21.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:20 smithi133 ceph-mon[29610]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:23:21.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:20 smithi188 ceph-mon[34645]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:23:23.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:23 smithi160 ceph-mon[34070]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:23:23.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:23 smithi133 ceph-mon[29610]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:23:23.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:23 smithi188 ceph-mon[34645]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:23:25.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:25 smithi160 ceph-mon[34070]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T08:23:25.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:25 smithi133 ceph-mon[29610]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T08:23:25.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:25 smithi188 ceph-mon[34645]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T08:23:26.875 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:26 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T08:23:26.875 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:26 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:26.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:26 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T08:23:26.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:26 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:26.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:26 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T08:23:26.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:26 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:27.888 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:27 smithi160 ceph-mon[34070]: Deploying daemon osd.4 on smithi160 2024-05-09T08:23:27.888 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:27 smithi160 ceph-mon[34070]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T08:23:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:27 smithi133 ceph-mon[29610]: Deploying daemon osd.4 on smithi160 2024-05-09T08:23:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:27 smithi133 ceph-mon[29610]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T08:23:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:27 smithi188 ceph-mon[34645]: Deploying daemon osd.4 on smithi160 2024-05-09T08:23:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:27 smithi188 ceph-mon[34645]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-09T08:23:29.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:29 smithi160 ceph-mon[34070]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:29 smithi133 ceph-mon[29610]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:29 smithi188 ceph-mon[34645]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:31 smithi160 ceph-mon[34070]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:31 smithi133 ceph-mon[29610]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:31 smithi188 ceph-mon[34645]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:33.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:33 smithi133 ceph-mon[29610]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:33.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:33 smithi188 ceph-mon[34645]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:33.937 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:33 smithi160 ceph-mon[34070]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:35.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.273+0000 7f218effd700 1 -- 172.21.15.160:0/1459304728 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2198062760 con 0x7f21840a2c20 2024-05-09T08:23:35.275 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 4 on host 'smithi160' 2024-05-09T08:23:35.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 msgr2=0x7f21840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 0x7f21840a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f218800b5c0 tx=0x7f2188005fb0 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 msgr2=0x7f21981986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f219400b9c0 tx=0x7f219400bd80 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 shutdown_connections 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f21840a2c20 0x7f21840a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2198103fb0 0x7f219819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2198100630 0x7f21981986e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 --2- 172.21.15.160:0/1459304728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2198103610 0x7f2198198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.276+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 >> 172.21.15.160:0/1459304728 conn(0x7f21980fbfb0 msgr2=0x7f21981069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:23:35.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.277+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 shutdown_connections 2024-05-09T08:23:35.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:23:35.277+0000 7f219fecf700 1 -- 172.21.15.160:0/1459304728 wait complete. 2024-05-09T08:23:35.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:35 smithi160 ceph-mon[34070]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:35.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:35 smithi160 ceph-mon[34070]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.908 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:35 smithi160 ceph-mon[34070]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:35 smithi133 ceph-mon[29610]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:35.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:35 smithi133 ceph-mon[29610]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:35 smithi133 ceph-mon[29610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:35 smithi188 ceph-mon[34645]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:35 smithi188 ceph-mon[34645]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:35 smithi188 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T08:23:35.935 DEBUG:teuthology.orchestra.run.smithi160:osd.4> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.4.service 2024-05-09T08:23:35.937 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-05-09T08:23:35.937 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T08:23:36.186 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.246 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:37 smithi160 ceph-mon[34070]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T08:23:37.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:37.334 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.334 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:37 smithi188 ceph-mon[34645]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:23:37.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:37 smithi133 ceph-mon[29610]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:37.607 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:23:37 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:23:37.244+0000 7fda82003700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T08:23:37.607 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:23:37 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:23:37.252+0000 7fda7ddf6700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114401 s, 917 MB/s 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 2024-05-09T08:23:38.290 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-09T08:23:38.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:38 smithi188 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:23:38.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:38 smithi188 ceph-mon[34645]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T08:23:38.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:38 smithi160 ceph-mon[34070]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:23:38.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:38 smithi160 ceph-mon[34070]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T08:23:38.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:38 smithi133 ceph-mon[29610]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:23:38.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:38 smithi133 ceph-mon[29610]: osdmap e33: 5 total, 4 up, 5 in 2024-05-09T08:23:38.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:38.929 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-05-09T08:23:39.168 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:23:39.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] boot 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:39 smithi160 ceph-mon[34070]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] boot 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T08:23:39.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.436 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:39 smithi188 ceph-mon[34645]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:39.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:23:39.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:23:39.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: osd.4 [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] boot 2024-05-09T08:23:39.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T08:23:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T08:23:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:39 smithi133 ceph-mon[29610]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi160 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-09T08:23:40.308 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi160 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:40.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: osdmap e35: 5 total, 5 up, 5 in 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi160 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 8544M 2024-05-09T08:23:40.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:23:40.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 -- 172.21.15.188:0/2795660422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc102e20 msgr2=0x7fcabc1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/2795660422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc102e20 0x7fcabc1096d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcab400b0a0 tx=0x7fcab400b3b0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 -- 172.21.15.188:0/2795660422 shutdown_connections 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/2795660422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc102e20 0x7fcabc1096d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/2795660422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc101280 0x7fcabc1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.842 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/2795660422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcabc102480 0x7fcabc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 -- 172.21.15.188:0/2795660422 >> 172.21.15.188:0/2795660422 conn(0x7fcabc0fc820 msgr2=0x7fcabc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 -- 172.21.15.188:0/2795660422 shutdown_connections 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.839+0000 7fcac2d2d700 1 -- 172.21.15.188:0/2795660422 wait complete. 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 Processor -- start 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 -- start start 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc102480 0x7fcabc19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcabc102e20 0x7fcabc19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10cf60 con 0x7fcabc102e20 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10cde0 con 0x7fcabc102480 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac2d2d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10d0e0 con 0x7fcabc101280 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:37636/0 (socket says 172.21.15.188:37636) 2024-05-09T08:23:40.843 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 -- 172.21.15.188:0/1339976764 learned_addr learned my addr 172.21.15.188:0/1339976764 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcabbfff700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc102480 0x7fcabc19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 -- 172.21.15.188:0/1339976764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc102480 msgr2=0x7fcabc19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc102480 0x7fcabc19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 -- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcabc102e20 msgr2=0x7fcabc19f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcabc102e20 0x7fcabc19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 -- 172.21.15.188:0/1339976764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc1a02c0 con 0x7fcabc101280 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcac0ac9700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b500 tx=0x7fcab000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0015020 con 0x7fcabc101280 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0011d10 con 0x7fcabc101280 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.840+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab000f2e0 con 0x7fcabc101280 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.841+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc1a05b0 con 0x7fcabc101280 2024-05-09T08:23:40.844 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.841+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a0b20 con 0x7fcabc101280 2024-05-09T08:23:40.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.842+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc069010 con 0x7fcabc101280 2024-05-09T08:23:40.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.843+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcab0018020 con 0x7fcabc101280 2024-05-09T08:23:40.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.843+0000 7fcab9ffb700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 0x7fcaac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:23:40.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.843+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fcab00cd7e0 con 0x7fcabc101280 2024-05-09T08:23:40.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.844+0000 7fcabbfff700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 0x7fcaac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:23:40.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.845+0000 7fcabbfff700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 0x7fcaac0a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006fd0 tx=0x7fcaa8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:23:40.849 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:40.848+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcab001f020 con 0x7fcabc101280 2024-05-09T08:23:41.052 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:23:41.051+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcabc062760 con 0x7fcaac0a2c20 2024-05-09T08:23:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:41 smithi188 ceph-mon[34645]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:41.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:41 smithi160 ceph-mon[34070]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:41.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:41.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:41 smithi133 ceph-mon[29610]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:41.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:23:41.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:23:41.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:42.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:42 smithi160 ceph-mon[34070]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:42.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:42 smithi133 ceph-mon[29610]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:42.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:42 smithi188 ceph-mon[34645]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:23:43.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:43 smithi160 ceph-mon[34070]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:43.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:43 smithi133 ceph-mon[29610]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:43 smithi188 ceph-mon[34645]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:44.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:44 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3541846200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:44 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:44 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]': finished 2024-05-09T08:23:44.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:44 smithi160 ceph-mon[34070]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T08:23:44.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:23:44.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:44 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3541846200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:44 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:44 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]': finished 2024-05-09T08:23:44.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:44 smithi133 ceph-mon[29610]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T08:23:44.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:23:44.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:44 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3541846200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:44 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]: dispatch 2024-05-09T08:23:44.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:44 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41748ba8-d09c-4d93-a815-11ef99a89358"}]': finished 2024-05-09T08:23:44.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:44 smithi188 ceph-mon[34645]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T08:23:44.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:23:45.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:45 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/4254448486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:45.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:45 smithi160 ceph-mon[34070]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:45.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:45 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/4254448486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:45.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:45 smithi133 ceph-mon[29610]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:45.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:45 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/4254448486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:23:45.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:45 smithi188 ceph-mon[34645]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:47.883 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:47 smithi188 ceph-mon[34645]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:47.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:47 smithi160 ceph-mon[34070]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:47.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:47 smithi133 ceph-mon[29610]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:49.831 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:49 smithi188 ceph-mon[34645]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:49.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:49 smithi160 ceph-mon[34070]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:49.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:49 smithi133 ceph-mon[29610]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:51.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:51 smithi160 ceph-mon[34070]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T08:23:51.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:51.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:51 smithi133 ceph-mon[29610]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:51.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T08:23:51.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:51.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:51 smithi188 ceph-mon[34645]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T08:23:51.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:23:52.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:52 smithi160 ceph-mon[34070]: Deploying daemon osd.5 on smithi188 2024-05-09T08:23:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:52 smithi133 ceph-mon[29610]: Deploying daemon osd.5 on smithi188 2024-05-09T08:23:52.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:52 smithi188 ceph-mon[34645]: Deploying daemon osd.5 on smithi188 2024-05-09T08:23:53.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:53 smithi133 ceph-mon[29610]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:53.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:53 smithi188 ceph-mon[34645]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:54.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:53 smithi160 ceph-mon[34070]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:55 smithi133 ceph-mon[29610]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:55 smithi188 ceph-mon[34645]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:55 smithi160 ceph-mon[34070]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:57.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:57 smithi188 ceph-mon[34645]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:57.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:57.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:57.803 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:57 smithi133 ceph-mon[29610]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:57.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:58.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:57 smithi160 ceph-mon[34070]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:58.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:58.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:23:58.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:23:59.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:23:59 smithi133 ceph-mon[29610]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:23:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:23:59 smithi188 ceph-mon[34645]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:00.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:23:59 smithi160 ceph-mon[34070]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:00.989 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.988+0000 7fcab9ffb700 1 -- 172.21.15.188:0/1339976764 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcabc062760 con 0x7fcaac0a2c20 2024-05-09T08:24:00.989 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 msgr2=0x7fcaac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 0x7fcaac0a50e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006fd0 tx=0x7fcaa8008040 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 msgr2=0x7fcabc06b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b500 tx=0x7fcab000b8c0 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 shutdown_connections 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcabc102e20 0x7fcabc19f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcaac0a2c20 0x7fcaac0a50e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcabc101280 0x7fcabc06b5b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 --2- 172.21.15.188:0/1339976764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcabc102480 0x7fcabc19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 >> 172.21.15.188:0/1339976764 conn(0x7fcabc0fc820 msgr2=0x7fcabc105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:24:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 shutdown_connections 2024-05-09T08:24:00.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:24:00.990+0000 7fcac2d2d700 1 -- 172.21.15.188:0/1339976764 wait complete. 2024-05-09T08:24:01.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:01 smithi188 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:01 smithi160 ceph-mon[34070]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:01 smithi133 ceph-mon[29610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T08:24:01.702 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.5.service 2024-05-09T08:24:01.704 INFO:tasks.cephadm:Deploying osd.6 on smithi133 with /dev/vg_nvme/lv_2... 2024-05-09T08:24:01.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T08:24:02.132 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.288 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:02 smithi133 ceph-mon[29610]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.291 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:02 smithi188 ceph-mon[34645]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T08:24:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: from='osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T08:24:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:24:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:02 smithi160 ceph-mon[34070]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-05-09T08:24:03.684 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:24:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:24:03.441+0000 7f1252176700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T08:24:03.684 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:24:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:24:03.448+0000 7f1249f61700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:24:04.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:24:04.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T08:24:04.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.090 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: from='osd.5 ' entity='osd.5' 2024-05-09T08:24:04.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] boot 2024-05-09T08:24:04.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T08:24:04.091 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:24:04.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T08:24:04.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: from='osd.5 ' entity='osd.5' 2024-05-09T08:24:04.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] boot 2024-05-09T08:24:04.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T08:24:04.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.430 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:24:04.430 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: osdmap e38: 6 total, 5 up, 6 in 2024-05-09T08:24:04.430 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: from='osd.5 ' entity='osd.5' 2024-05-09T08:24:04.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: osd.5 [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] boot 2024-05-09T08:24:04.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T08:24:04.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T08:24:04.969 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T08:24:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T08:24:04.970 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-05-09T08:24:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-05-09T08:24:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0336953 s, 311 MB/s 2024-05-09T08:24:04.970 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:05.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:04 smithi133 ceph-mon[29610]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T08:24:05.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:24:05.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:04 smithi188 ceph-mon[34645]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T08:24:05.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:05.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:05.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:05.415 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:04 smithi160 ceph-mon[34070]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T08:24:05.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-05-09T08:24:06.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:05 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi188 2024-05-09T08:24:06.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:05 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-09T08:24:06.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:05 smithi133 ceph-mon[29610]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T08:24:06.240 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:24:06.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:05 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi188 2024-05-09T08:24:06.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:05 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-09T08:24:06.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:05 smithi160 ceph-mon[34070]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T08:24:06.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:05 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi188 2024-05-09T08:24:06.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:05 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 8544M 2024-05-09T08:24:06.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:05 smithi188 ceph-mon[34645]: osdmap e41: 6 total, 6 up, 6 in 2024-05-09T08:24:07.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:07 smithi133 ceph-mon[29610]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:07.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:07 smithi133 ceph-mon[29610]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T08:24:07.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:07 smithi160 ceph-mon[34070]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:07.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:07 smithi160 ceph-mon[34070]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T08:24:07.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:07 smithi188 ceph-mon[34645]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:07.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:07 smithi188 ceph-mon[34645]: osdmap e42: 6 total, 6 up, 6 in 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.519+0000 7f9146800700 1 -- 172.21.15.133:0/2144359802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 msgr2=0x7f914010a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.519+0000 7f9146800700 1 --2- 172.21.15.133:0/2144359802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914010a020 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91280099c0 tx=0x7f9128009cd0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 -- 172.21.15.133:0/2144359802 shutdown_connections 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 --2- 172.21.15.133:0/2144359802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f9140110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 --2- 172.21.15.133:0/2144359802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 0x7f9140108de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 --2- 172.21.15.133:0/2144359802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914010a020 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 -- 172.21.15.133:0/2144359802 >> 172.21.15.133:0/2144359802 conn(0x7f9140077990 msgr2=0x7f9140079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:24:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 -- 172.21.15.133:0/2144359802 shutdown_connections 2024-05-09T08:24:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.520+0000 7f9146800700 1 -- 172.21.15.133:0/2144359802 wait complete. 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 Processor -- start 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 -- start start 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 0x7f914019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f91401a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91401145a0 con 0x7f9140109bc0 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9140114420 con 0x7f914010a560 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9146800700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9140114720 con 0x7f91401089c0 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.522+0000 7f9136a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f9136a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34602/0 (socket says 172.21.15.133:34602) 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f91401a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f9136a00700 1 -- 172.21.15.133:0/3347187691 learned_addr learned my addr 172.21.15.133:0/3347187691 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913f400700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 0x7f914019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 -- 172.21.15.133:0/3347187691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 msgr2=0x7f914019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 0x7f914019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 -- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 msgr2=0x7f914019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 -- 172.21.15.133:0/3347187691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401a67b0 con 0x7f914010a560 2024-05-09T08:24:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913fe00700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f91401a6110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f913800e970 tx=0x7f913800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.523+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138004220 con 0x7f914010a560 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.524+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9138004d10 con 0x7f914010a560 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.524+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91401a6aa0 con 0x7f914010a560 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.524+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138010700 con 0x7f914010a560 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.524+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a6f90 con 0x7f914010a560 2024-05-09T08:24:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.525+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140069010 con 0x7f914010a560 2024-05-09T08:24:08.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.527+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9138004750 con 0x7f914010a560 2024-05-09T08:24:08.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.528+0000 7f913cc00700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.528+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f91380cdf60 con 0x7f914010a560 2024-05-09T08:24:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.528+0000 7f913f400700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.529+0000 7f913f400700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 0x7f91240a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9140109a20 tx=0x7f913000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:24:08.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.532+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f913809bce0 con 0x7f914010a560 2024-05-09T08:24:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:08.765+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9140062760 con 0x7f91240a2c20 2024-05-09T08:24:09.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:08 smithi160 ceph-mon[34070]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T08:24:09.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:08 smithi160 ceph-mon[34070]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:24:09.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:09.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:09.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:08 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:09.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:08 smithi133 ceph-mon[29610]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T08:24:09.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:08 smithi133 ceph-mon[29610]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:24:09.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:08 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:09.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:08 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:09.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:08 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:09.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:08 smithi188 ceph-mon[34645]: osdmap e43: 6 total, 6 up, 6 in 2024-05-09T08:24:09.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:08 smithi188 ceph-mon[34645]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 184 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:24:09.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:08 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:09.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:08 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:09.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:08 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:09.974 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:09 smithi133 ceph-mon[29610]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:10.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:09 smithi160 ceph-mon[34070]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:10.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:09 smithi188 ceph-mon[34645]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:11.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:10 smithi160 ceph-mon[34070]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T08:24:11.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:10 smithi133 ceph-mon[29610]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T08:24:11.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:10 smithi188 ceph-mon[34645]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-09T08:24:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:13 smithi160 ceph-mon[34070]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T08:24:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:13 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]: dispatch 2024-05-09T08:24:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:13 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]': finished 2024-05-09T08:24:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:13 smithi160 ceph-mon[34070]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T08:24:13.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:13 smithi133 ceph-mon[29610]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T08:24:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:13 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]: dispatch 2024-05-09T08:24:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:13 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]': finished 2024-05-09T08:24:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:13 smithi133 ceph-mon[29610]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T08:24:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:13 smithi188 ceph-mon[34645]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-09T08:24:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:13 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]: dispatch 2024-05-09T08:24:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:13 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1942914323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc1a882a-80f7-4af1-954d-4005e2b92111"}]': finished 2024-05-09T08:24:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:13 smithi188 ceph-mon[34645]: osdmap e44: 7 total, 6 up, 7 in 2024-05-09T08:24:13.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:14.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:14 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2396492378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:14.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:14 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2396492378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:14.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:14 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2396492378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:15.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:15 smithi133 ceph-mon[29610]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-09T08:24:15.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:15 smithi188 ceph-mon[34645]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-09T08:24:16.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:15 smithi160 ceph-mon[34070]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-09T08:24:17.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:17 smithi133 ceph-mon[29610]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:17.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:17 smithi188 ceph-mon[34645]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:17.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:17 smithi160 ceph-mon[34070]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:19.876 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:19 smithi133 ceph-mon[29610]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:19.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:19 smithi188 ceph-mon[34645]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:19 smithi160 ceph-mon[34070]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:21.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:21 smithi133 ceph-mon[29610]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:21.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:21 smithi188 ceph-mon[34645]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:21 smithi160 ceph-mon[34070]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:22.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T08:24:22.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:23.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T08:24:23.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:23.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T08:24:23.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:23.782 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:23 smithi133 ceph-mon[29610]: Deploying daemon osd.6 on smithi133 2024-05-09T08:24:23.783 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:23 smithi133 ceph-mon[29610]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:24.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:23 smithi160 ceph-mon[34070]: Deploying daemon osd.6 on smithi133 2024-05-09T08:24:24.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:23 smithi160 ceph-mon[34070]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:24.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:23 smithi188 ceph-mon[34645]: Deploying daemon osd.6 on smithi133 2024-05-09T08:24:24.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:23 smithi188 ceph-mon[34645]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:25.840 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:25 smithi133 ceph-mon[29610]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:26.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:25 smithi160 ceph-mon[34070]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:26.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:25 smithi188 ceph-mon[34645]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:27.824 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:27 smithi133 ceph-mon[29610]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:28.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:27 smithi160 ceph-mon[34070]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:28.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:27 smithi188 ceph-mon[34645]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:28.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:28 smithi133 ceph-mon[29610]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:29.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:28 smithi160 ceph-mon[34070]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:29.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:28 smithi188 ceph-mon[34645]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:31 smithi160 ceph-mon[34070]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:31.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:31 smithi133 ceph-mon[29610]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:31.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:31 smithi188 ceph-mon[34645]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:33.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:33 smithi160 ceph-mon[34070]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:33.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:33 smithi133 ceph-mon[29610]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:33.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:33 smithi188 ceph-mon[34645]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:35.828 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:35 smithi133 ceph-mon[29610]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:35.828 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.828 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.829 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:35 smithi133 ceph-mon[29610]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T08:24:35.910 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 6 on host 'smithi133' 2024-05-09T08:24:35.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.907+0000 7f913cc00700 1 -- 172.21.15.133:0/3347187691 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9140062760 con 0x7f91240a2c20 2024-05-09T08:24:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:35 smithi160 ceph-mon[34070]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:35 smithi160 ceph-mon[34070]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 msgr2=0x7f91240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 0x7f91240a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9140109a20 tx=0x7f913000b040 comp rx=0 tx=0).stop 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 msgr2=0x7f91401a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f91401a6110 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f913800e970 tx=0x7f913800ec80 comp rx=0 tx=0).stop 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 shutdown_connections 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f914010a560 0x7f91401a6110 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:35.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91401089c0 0x7f914019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 --2- 172.21.15.133:0/3347187691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9140109bc0 0x7f914019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 >> 172.21.15.133:0/3347187691 conn(0x7f9140077990 msgr2=0x7f914010ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:24:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 shutdown_connections 2024-05-09T08:24:35.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:24:35.912+0000 7f9146800700 1 -- 172.21.15.133:0/3347187691 wait complete. 2024-05-09T08:24:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:35 smithi188 ceph-mon[34645]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:35 smithi188 ceph-mon[34645]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T08:24:36.701 DEBUG:teuthology.orchestra.run.smithi133:osd.6> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.6.service 2024-05-09T08:24:36.708 INFO:tasks.cephadm:Deploying osd.7 on smithi160 with /dev/vg_nvme/lv_2... 2024-05-09T08:24:36.708 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T08:24:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T08:24:36.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T08:24:36.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:24:36.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:36.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:36 smithi133 ceph-mon[29610]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:36.954 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:24:37.075 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:37.075 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:37.076 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T08:24:37.076 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T08:24:37.076 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:24:37.076 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:37.076 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:36 smithi160 ceph-mon[34070]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:37.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:37.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:37.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T08:24:37.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: osdmap e45: 7 total, 6 up, 7 in 2024-05-09T08:24:37.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:24:37.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:37.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:36 smithi188 ceph-mon[34645]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:37.493 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:24:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:24:37.217+0000 7fdb89200700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T08:24:37.494 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:24:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:24:37.230+0000 7fdb84000700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:24:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:38 smithi160 ceph-mon[34070]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:24:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:38 smithi160 ceph-mon[34070]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T08:24:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:38.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:38.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:38 smithi133 ceph-mon[29610]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:24:38.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:38 smithi133 ceph-mon[29610]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T08:24:38.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:38.506 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:38.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:38 smithi188 ceph-mon[34645]: from='osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:24:38.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:38 smithi188 ceph-mon[34645]: osdmap e46: 7 total, 6 up, 7 in 2024-05-09T08:24:38.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:38.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.016 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T08:24:39.016 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T08:24:39.016 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-09T08:24:39.017 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-09T08:24:39.017 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116449 s, 900 MB/s 2024-05-09T08:24:39.017 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-09T08:24:39.485 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:24:39.485 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:24:39.485 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.486 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] boot 2024-05-09T08:24:39.486 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T08:24:39.486 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.486 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:39 smithi160 ceph-mon[34070]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] boot 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.585 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:39 smithi188 ceph-mon[34645]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:39.654 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: osd.6 [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] boot 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: osdmap e47: 7 total, 7 up, 7 in 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T08:24:39.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:39 smithi133 ceph-mon[29610]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T08:24:39.897 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:24:40.277 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:40 smithi160 ceph-mon[34070]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T08:24:40.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:40 smithi133 ceph-mon[29610]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T08:24:40.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:40 smithi188 ceph-mon[34645]: osdmap e48: 7 total, 7 up, 7 in 2024-05-09T08:24:41.295 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:41 smithi160 ceph-mon[34070]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 -- 172.21.15.160:0/1875635741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 msgr2=0x7f59e810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 --2- 172.21.15.160:0/1875635741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e810a7d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f59e000b0d0 tx=0x7f59e000b3e0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 -- 172.21.15.160:0/1875635741 shutdown_connections 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 --2- 172.21.15.160:0/1875635741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e810a7d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 --2- 172.21.15.160:0/1875635741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8102380 0x7f59e81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 --2- 172.21.15.160:0/1875635741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e8103580 0x7f59e81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.529+0000 7f59eef3d700 1 -- 172.21.15.160:0/1875635741 >> 172.21.15.160:0/1875635741 conn(0x7f59e80fd920 msgr2=0x7f59e80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:24:41.533 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- 172.21.15.160:0/1875635741 shutdown_connections 2024-05-09T08:24:41.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- 172.21.15.160:0/1875635741 wait complete. 2024-05-09T08:24:41.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 Processor -- start 2024-05-09T08:24:41.534 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- start start 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e8102380 0x7f59e81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 0x7f59e8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810e060 con 0x7f59e8102380 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810dee0 con 0x7f59e8103f20 2024-05-09T08:24:41.536 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59eef3d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810e1e0 con 0x7f59e8103580 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59ed4da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59ed4da700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:36100/0 (socket says 172.21.15.160:36100) 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.530+0000 7f59ed4da700 1 -- 172.21.15.160:0/2430970674 learned_addr learned my addr 172.21.15.160:0/2430970674 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59e7fff700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 0x7f59e8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 -- 172.21.15.160:0/2430970674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 msgr2=0x7f59e8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 0x7f59e8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 -- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e8102380 msgr2=0x7f59e81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e8102380 0x7f59e81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 -- 172.21.15.160:0/2430970674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e81a0300 con 0x7f59e8103f20 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59e7fff700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 0x7f59e8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59ed4da700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59e00062a0 tx=0x7f59e0007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0003c00 con 0x7f59e8103f20 2024-05-09T08:24:41.537 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81a0590 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e0003da0 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0005560 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.531+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a0af0 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.532+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e8069010 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.533+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f59e0005700 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.534+0000 7f59e5ffb700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 0x7f59d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.534+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f59e000f020 con 0x7f59e8103f20 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.534+0000 7f59eccd9700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 0x7f59d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:24:41.538 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.535+0000 7f59eccd9700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 0x7f59d80a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e81033e0 tx=0x7f59dc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:24:41.539 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.538+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f59e002b020 con 0x7f59e8103f20 2024-05-09T08:24:41.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:41 smithi133 ceph-mon[29610]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:41.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:41 smithi188 ceph-mon[34645]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:41.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:24:41.745+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f59e8062760 con 0x7f59d80a2c20 2024-05-09T08:24:42.524 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:42.525 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:42.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:42.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:42.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:42.685 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi133 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 5696M 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:43.606 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:43 smithi160 ceph-mon[34070]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:43.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi133 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 5696M 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:43.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:43 smithi133 ceph-mon[29610]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:24:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi133 2024-05-09T08:24:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 5696M 2024-05-09T08:24:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:24:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:43 smithi188 ceph-mon[34645]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:44.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:44 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/4055520672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:44 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:44 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]': finished 2024-05-09T08:24:44.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:44 smithi160 ceph-mon[34070]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T08:24:44.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:24:44.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:44 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/4055520672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:44 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:44 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]': finished 2024-05-09T08:24:44.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:44 smithi133 ceph-mon[29610]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T08:24:44.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:24:44.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:44 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/4055520672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:44 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]: dispatch 2024-05-09T08:24:44.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:44 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4a0b43-4191-4999-93e4-929b414f0822"}]': finished 2024-05-09T08:24:44.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:44 smithi188 ceph-mon[34645]: osdmap e49: 8 total, 7 up, 8 in 2024-05-09T08:24:44.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:24:45.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:45 smithi160 ceph-mon[34070]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:45.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:45 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/2845333011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:45.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:45 smithi133 ceph-mon[29610]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:45.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:45 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/2845333011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:45.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:45 smithi188 ceph-mon[34645]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:45.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:45 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/2845333011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:24:47.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:47 smithi160 ceph-mon[34070]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:47.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:47 smithi133 ceph-mon[29610]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:47.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:47 smithi188 ceph-mon[34645]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:49.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:49 smithi160 ceph-mon[34070]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:49.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:49 smithi133 ceph-mon[29610]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:49.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:49 smithi188 ceph-mon[34645]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:51.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:51 smithi160 ceph-mon[34070]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:51.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:51 smithi133 ceph-mon[29610]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:51.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:51 smithi188 ceph-mon[34645]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:52.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T08:24:52.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T08:24:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:52.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T08:24:52.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:24:53.923 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:53 smithi160 ceph-mon[34070]: Deploying daemon osd.7 on smithi160 2024-05-09T08:24:53.924 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:53 smithi160 ceph-mon[34070]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:53.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:53 smithi133 ceph-mon[29610]: Deploying daemon osd.7 on smithi160 2024-05-09T08:24:53.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:53 smithi133 ceph-mon[29610]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:53.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:53 smithi188 ceph-mon[34645]: Deploying daemon osd.7 on smithi160 2024-05-09T08:24:53.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:53 smithi188 ceph-mon[34645]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:55.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:55 smithi133 ceph-mon[29610]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:55 smithi188 ceph-mon[34645]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:55.942 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:55 smithi160 ceph-mon[34070]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:57.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:57 smithi160 ceph-mon[34070]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:57 smithi133 ceph-mon[29610]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:57 smithi188 ceph-mon[34645]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:58.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:58.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:58.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:58 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:24:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:58 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:59.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:58 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:24:59.767 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:24:59 smithi160 ceph-mon[34070]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:59.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:24:59 smithi133 ceph-mon[29610]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:24:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:24:59 smithi188 ceph-mon[34645]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:01.382 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 7 on host 'smithi160' 2024-05-09T08:25:01.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.381+0000 7f59e5ffb700 1 -- 172.21.15.160:0/2430970674 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59e8062760 con 0x7f59d80a2c20 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 msgr2=0x7f59d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 0x7f59d80a50e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f59e81033e0 tx=0x7f59dc006d20 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 msgr2=0x7f59e819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f59e00062a0 tx=0x7f59e0007e40 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 shutdown_connections 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59e8103f20 0x7f59e819fc60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59e8102380 0x7f59e81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f59d80a2c20 0x7f59d80a50e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 --2- 172.21.15.160:0/2430970674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59e8103580 0x7f59e8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 >> 172.21.15.160:0/2430970674 conn(0x7f59e80fd920 msgr2=0x7f59e81068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 shutdown_connections 2024-05-09T08:25:01.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:25:01.384+0000 7f59eef3d700 1 -- 172.21.15.160:0/2430970674 wait complete. 2024-05-09T08:25:01.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:01 smithi160 ceph-mon[34070]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:01.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:01.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:01.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:01 smithi160 ceph-mon[34070]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:01.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:01 smithi160 ceph-mon[34070]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:02.017 DEBUG:teuthology.orchestra.run.smithi160:osd.7> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.7.service 2024-05-09T08:25:02.020 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-05-09T08:25:02.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T08:25:02.046 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:01 smithi188 ceph-mon[34645]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:02.046 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:02.046 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:02.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:01 smithi188 ceph-mon[34645]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:02.047 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:01 smithi188 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:02.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:01 smithi133 ceph-mon[29610]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:02.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:02.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:02.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:01 smithi133 ceph-mon[29610]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:02.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:01 smithi133 ceph-mon[29610]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T08:25:02.294 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:25:03.370 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.371 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:03 smithi188 ceph-mon[34645]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:03.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:03 smithi160 ceph-mon[34070]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:03.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:03.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T08:25:03.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: osdmap e50: 8 total, 7 up, 8 in 2024-05-09T08:25:03.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:03.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:25:03.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:03 smithi133 ceph-mon[29610]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T08:25:04.070 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:25:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:25:03.789+0000 7f4529842700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T08:25:04.070 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:25:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:25:03.799+0000 7f452162d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113436 s, 924 MB/s 2024-05-09T08:25:04.244 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-09T08:25:04.551 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:04 smithi188 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:25:04.551 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:04 smithi188 ceph-mon[34645]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T08:25:04.551 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:04.551 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:04 smithi188 ceph-mon[34645]: from='osd.7 ' entity='osd.7' 2024-05-09T08:25:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:04 smithi160 ceph-mon[34070]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:25:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:04 smithi160 ceph-mon[34070]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T08:25:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:04 smithi160 ceph-mon[34070]: from='osd.7 ' entity='osd.7' 2024-05-09T08:25:04.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:04 smithi133 ceph-mon[29610]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:25:04.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:04 smithi133 ceph-mon[29610]: osdmap e51: 8 total, 7 up, 8 in 2024-05-09T08:25:04.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:04.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:04 smithi133 ceph-mon[29610]: from='osd.7 ' entity='osd.7' 2024-05-09T08:25:04.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-05-09T08:25:05.254 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] boot 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:05.481 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:05 smithi160 ceph-mon[34070]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] boot 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:05.648 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:05 smithi188 ceph-mon[34645]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: osd.7 [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] boot 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: osdmap e52: 8 total, 8 up, 8 in 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: pgmap v196: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:05 smithi133 ceph-mon[29610]: osdmap e53: 8 total, 8 up, 8 in 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi160 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.644 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.645 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:06 smithi188 ceph-mon[34645]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi160 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:06 smithi160 ceph-mon[34070]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T08:25:06.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-09T08:25:06.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi160 2024-05-09T08:25:06.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 5696M 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:06.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:06 smithi133 ceph-mon[29610]: osdmap e54: 8 total, 8 up, 8 in 2024-05-09T08:25:06.977 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 -- 172.21.15.188:0/1498179943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102480 msgr2=0x7f3b241028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 --2- 172.21.15.188:0/1498179943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102480 0x7f3b241028e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009990 tx=0x7f3b20009ca0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 -- 172.21.15.188:0/1498179943 shutdown_connections 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 --2- 172.21.15.188:0/1498179943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102e20 0x7f3b241096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 --2- 172.21.15.188:0/1498179943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b24101280 0x7f3b241016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 --2- 172.21.15.188:0/1498179943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102480 0x7f3b241028e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 -- 172.21.15.188:0/1498179943 >> 172.21.15.188:0/1498179943 conn(0x7f3b240fc840 msgr2=0x7f3b240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 -- 172.21.15.188:0/1498179943 shutdown_connections 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.975+0000 7f3b2befa700 1 -- 172.21.15.188:0/1498179943 wait complete. 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 Processor -- start 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 -- start start 2024-05-09T08:25:06.978 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b24101280 0x7f3b2406b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102480 0x7f3b2419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410cf60 con 0x7f3b24101280 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410cde0 con 0x7f3b24102480 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2befa700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2410d0e0 con 0x7f3b24102e20 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:41120/0 (socket says 172.21.15.188:41120) 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 -- 172.21.15.188:0/2105818805 learned_addr learned my addr 172.21.15.188:0/2105818805 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 -- 172.21.15.188:0/2105818805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102480 msgr2=0x7f3b2419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b29495700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102480 0x7f3b2419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102480 0x7f3b2419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 -- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b24101280 msgr2=0x7f3b2406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b24101280 0x7f3b2406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 -- 172.21.15.188:0/2105818805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b241a0350 con 0x7f3b24102e20 2024-05-09T08:25:06.979 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.976+0000 7f3b2a497700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00b970 tx=0x7f3b1c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:25:06.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.977+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c019070 con 0x7f3b24102e20 2024-05-09T08:25:06.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.977+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c014440 con 0x7f3b24102e20 2024-05-09T08:25:06.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.977+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241a0640 con 0x7f3b24102e20 2024-05-09T08:25:06.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.977+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0075d0 con 0x7f3b24102e20 2024-05-09T08:25:06.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.977+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a0bb0 con 0x7f3b24102e20 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.978+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b24069010 con 0x7f3b24102e20 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.979+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b1c01c070 con 0x7f3b24102e20 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.979+0000 7f3b1affd700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 0x7f3b100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.979+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f3b1c0cde10 con 0x7f3b24102e20 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.980+0000 7f3b29c96700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 0x7f3b100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:06.981 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.981+0000 7f3b29c96700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 0x7f3b100a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b241022e0 tx=0x7f3b14006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:25:06.983 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:06.983+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b1c09b8d0 con 0x7f3b24102e20 2024-05-09T08:25:07.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:07.187+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3b24062760 con 0x7f3b100a2c20 2024-05-09T08:25:07.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:07 smithi160 ceph-mon[34070]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:07.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:07 smithi160 ceph-mon[34070]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T08:25:07.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:07.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:07.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:07.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:07 smithi133 ceph-mon[29610]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:07.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:07 smithi133 ceph-mon[29610]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T08:25:07.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:07.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:07.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:07.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:07 smithi188 ceph-mon[34645]: pgmap v199: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:07.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:07 smithi188 ceph-mon[34645]: osdmap e55: 8 total, 8 up, 8 in 2024-05-09T08:25:07.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:07.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:07.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:08.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:08 smithi160 ceph-mon[34070]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:08.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:08 smithi160 ceph-mon[34070]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T08:25:08.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:08 smithi133 ceph-mon[29610]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:08.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:08 smithi133 ceph-mon[29610]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T08:25:08.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:08 smithi188 ceph-mon[34645]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:08.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:08 smithi188 ceph-mon[34645]: osdmap e56: 8 total, 8 up, 8 in 2024-05-09T08:25:09.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:09 smithi160 ceph-mon[34070]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:09.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:09 smithi133 ceph-mon[29610]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:09.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:09 smithi188 ceph-mon[34645]: pgmap v202: 1 pgs: 1 unknown; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:10.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:10 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/1707208576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:10 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:10 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]': finished 2024-05-09T08:25:10.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:10 smithi160 ceph-mon[34070]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T08:25:10.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:10.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:10 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/1707208576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:10 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:10 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]': finished 2024-05-09T08:25:10.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:10 smithi133 ceph-mon[29610]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T08:25:10.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:10.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:10 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/1707208576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:10 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]: dispatch 2024-05-09T08:25:10.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:10 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbb1ac-8999-488b-965a-a1d6236162f3"}]': finished 2024-05-09T08:25:10.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:10 smithi188 ceph-mon[34645]: osdmap e57: 9 total, 8 up, 9 in 2024-05-09T08:25:10.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:11.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:11 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3605389154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:11.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:11 smithi133 ceph-mon[29610]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:11.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:11 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3605389154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:11.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:11 smithi188 ceph-mon[34645]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:11.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:11 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3605389154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:11.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:11 smithi160 ceph-mon[34070]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:12 smithi133 ceph-mon[29610]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-09T08:25:12.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:12 smithi133 ceph-mon[29610]: Cluster is now healthy 2024-05-09T08:25:12.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:12 smithi188 ceph-mon[34645]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-09T08:25:12.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:12 smithi188 ceph-mon[34645]: Cluster is now healthy 2024-05-09T08:25:12.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:12 smithi160 ceph-mon[34070]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-09T08:25:12.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:12 smithi160 ceph-mon[34070]: Cluster is now healthy 2024-05-09T08:25:13.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:13 smithi133 ceph-mon[29610]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:13.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:13 smithi188 ceph-mon[34645]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:13.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:13 smithi160 ceph-mon[34070]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:15.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:15 smithi133 ceph-mon[29610]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:15.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:15 smithi188 ceph-mon[34645]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:16.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:15 smithi160 ceph-mon[34070]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:17.671 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:17 smithi188 ceph-mon[34645]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:17.671 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T08:25:17.671 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:17 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:17.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:17 smithi133 ceph-mon[29610]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:17.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T08:25:17.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:17 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:17.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:17 smithi160 ceph-mon[34070]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:17.966 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T08:25:17.966 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:17 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:18.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:18 smithi188 ceph-mon[34645]: Deploying daemon osd.8 on smithi188 2024-05-09T08:25:18.982 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:18 smithi160 ceph-mon[34070]: Deploying daemon osd.8 on smithi188 2024-05-09T08:25:19.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:18 smithi133 ceph-mon[29610]: Deploying daemon osd.8 on smithi188 2024-05-09T08:25:19.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:19 smithi188 ceph-mon[34645]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:19 smithi160 ceph-mon[34070]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:19 smithi133 ceph-mon[29610]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:21.912 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:21 smithi188 ceph-mon[34645]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:21 smithi160 ceph-mon[34070]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:22.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:21 smithi133 ceph-mon[29610]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:23.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:23 smithi188 ceph-mon[34645]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:23.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:23.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:23.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:24.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:23 smithi160 ceph-mon[34070]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:24.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:24.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:24.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:24.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:23 smithi133 ceph-mon[29610]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:24.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:24.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:24.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:25.950 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:25 smithi188 ceph-mon[34645]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:26.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:25 smithi160 ceph-mon[34070]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:26.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:25 smithi133 ceph-mon[29610]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:27.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.076+0000 7f3b1affd700 1 -- 172.21.15.188:0/2105818805 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b24062760 con 0x7f3b100a2c20 2024-05-09T08:25:27.076 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-05-09T08:25:27.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.077+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 msgr2=0x7f3b100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:27.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.077+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 0x7f3b100a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3b241022e0 tx=0x7f3b14006d20 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 msgr2=0x7f3b2419fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00b970 tx=0x7f3b1c00bd30 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 shutdown_connections 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2c20 0x7f3b100a50e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24102e20 0x7f3b2419fa80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b24101280 0x7f3b2406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 --2- 172.21.15.188:0/2105818805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24102480 0x7f3b2419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 >> 172.21.15.188:0/2105818805 conn(0x7f3b240fc840 msgr2=0x7f3b24105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 shutdown_connections 2024-05-09T08:25:27.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:25:27.078+0000 7f3b2befa700 1 -- 172.21.15.188:0/2105818805 wait complete. 2024-05-09T08:25:27.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:27.400 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.401 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:27.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:27.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.665 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.666 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:27.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:27.680 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T08:25:27.681 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:27.723 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.8.service 2024-05-09T08:25:27.727 INFO:tasks.cephadm:Deploying osd.9 on smithi133 with /dev/vg_nvme/lv_1... 2024-05-09T08:25:27.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T08:25:28.166 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:29 smithi188 ceph-mon[34645]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:29.385 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T08:25:29.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T08:25:29.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:29.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.386 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:29 smithi133 ceph-mon[29610]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:29.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T08:25:29.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: from='osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: osdmap e58: 9 total, 8 up, 9 in 2024-05-09T08:25:29.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:29.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:25:29.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:29 smithi160 ceph-mon[34070]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:29.933 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:25:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:25:29.502+0000 7faff3da0700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T08:25:29.933 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:25:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:25:29.513+0000 7fafeb38a700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:30.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:30 smithi133 ceph-mon[29610]: from='osd.8 ' entity='osd.8' 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:30.365 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:30 smithi188 ceph-mon[34645]: from='osd.8 ' entity='osd.8' 2024-05-09T08:25:30.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:25:30.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:25:30.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:25:30.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: osdmap e59: 9 total, 8 up, 9 in 2024-05-09T08:25:30.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:30.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:30 smithi160 ceph-mon[34070]: from='osd.8 ' entity='osd.8' 2024-05-09T08:25:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T08:25:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T08:25:31.232 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-05-09T08:25:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-05-09T08:25:31.232 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363893 s, 288 MB/s 2024-05-09T08:25:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-05-09T08:25:31.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] boot 2024-05-09T08:25:31.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T08:25:31.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:31 smithi160 ceph-mon[34070]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T08:25:31.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] boot 2024-05-09T08:25:31.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T08:25:31.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:31 smithi133 ceph-mon[29610]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T08:25:31.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: osd.8 [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] boot 2024-05-09T08:25:31.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: osdmap e60: 9 total, 9 up, 9 in 2024-05-09T08:25:31.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:25:31.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:31.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:31 smithi188 ceph-mon[34645]: osdmap e61: 9 total, 9 up, 9 in 2024-05-09T08:25:32.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-05-09T08:25:32.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:32 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi188 2024-05-09T08:25:32.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:32 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-09T08:25:32.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:32 smithi160 ceph-mon[34070]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T08:25:32.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:32 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi188 2024-05-09T08:25:32.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:32 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-09T08:25:32.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:32 smithi133 ceph-mon[29610]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T08:25:32.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:32 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi188 2024-05-09T08:25:32.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:32 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 5696M 2024-05-09T08:25:32.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:32 smithi188 ceph-mon[34645]: osdmap e62: 9 total, 9 up, 9 in 2024-05-09T08:25:32.495 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:25:33.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:33 smithi160 ceph-mon[34070]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:33 smithi133 ceph-mon[29610]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:33.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:33 smithi188 ceph-mon[34645]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:34.390 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:34 smithi133 ceph-mon[29610]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T08:25:34.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:34 smithi160 ceph-mon[34070]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T08:25:34.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:34 smithi188 ceph-mon[34645]: osdmap e63: 9 total, 9 up, 9 in 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 -- 172.21.15.133:0/2298401610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44540fffe0 msgr2=0x7f4454100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 --2- 172.21.15.133:0/2298401610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44540fffe0 0x7f4454100400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f44440098f0 tx=0x7f4444009c00 comp rx=0 tx=0).stop 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 -- 172.21.15.133:0/2298401610 shutdown_connections 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 --2- 172.21.15.133:0/2298401610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4454108060 0x7f445410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 --2- 172.21.15.133:0/2298401610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44540fffe0 0x7f4454100400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 --2- 172.21.15.133:0/2298401610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 0x7f4454107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 -- 172.21.15.133:0/2298401610 >> 172.21.15.133:0/2298401610 conn(0x7f44540fbb70 msgr2=0x7f44540fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 -- 172.21.15.133:0/2298401610 shutdown_connections 2024-05-09T08:25:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.681+0000 7f4459200700 1 -- 172.21.15.133:0/2298401610 wait complete. 2024-05-09T08:25:34.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 Processor -- start 2024-05-09T08:25:34.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 -- start start 2024-05-09T08:25:34.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:34.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 0x7f445419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4454108060 0x7f44541a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f445410df70 con 0x7f44540fffe0 2024-05-09T08:25:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f445410ddf0 con 0x7f4454108060 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4459200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f445410e0f0 con 0x7f44541009d0 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.683+0000 7f4451e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33280/0 (socket says 172.21.15.133:33280) 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 -- 172.21.15.133:0/3749451830 learned_addr learned my addr 172.21.15.133:0/3749451830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f444be00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 0x7f445419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4452800700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4454108060 0x7f44541a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 -- 172.21.15.133:0/3749451830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 msgr2=0x7f445419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 0x7f445419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 -- 172.21.15.133:0/3749451830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4454108060 msgr2=0x7f44541a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4454108060 0x7f44541a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 -- 172.21.15.133:0/3749451830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44541a2430 con 0x7f44540fffe0 2024-05-09T08:25:34.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.684+0000 7f4451e00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f444400fca0 tx=0x7f4444005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.685+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444025070 con 0x7f44540fffe0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.685+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4444005650 con 0x7f44540fffe0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.685+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44541a26c0 con 0x7f44540fffe0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.685+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444003c70 con 0x7f44540fffe0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.685+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44541a2b10 con 0x7f44540fffe0 2024-05-09T08:25:34.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.686+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4454069010 con 0x7f44540fffe0 2024-05-09T08:25:34.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.689+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4444028070 con 0x7f44540fffe0 2024-05-09T08:25:34.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.691+0000 7f444aa00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 0x7f44380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:25:34.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.691+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f44440cf440 con 0x7f44540fffe0 2024-05-09T08:25:34.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.691+0000 7f444be00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 0x7f44380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:25:34.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.692+0000 7f444be00700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 0x7f44380a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f443c009c60 tx=0x7f443c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:25:34.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.692+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f444402f020 con 0x7f44540fffe0 2024-05-09T08:25:34.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:25:34.927+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f44540fd6d0 con 0x7f44380a2c20 2024-05-09T08:25:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:35 smithi160 ceph-mon[34070]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T08:25:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:35 smithi160 ceph-mon[34070]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:25:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:35 smithi133 ceph-mon[29610]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T08:25:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:35 smithi133 ceph-mon[29610]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:25:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:35.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:35 smithi188 ceph-mon[34645]: osdmap e64: 9 total, 9 up, 9 in 2024-05-09T08:25:35.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:35 smithi188 ceph-mon[34645]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:25:35.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:25:35.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:25:35.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:36.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:36 smithi160 ceph-mon[34070]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:36 smithi133 ceph-mon[29610]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:36.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:36 smithi188 ceph-mon[34645]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:25:37.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:37 smithi133 ceph-mon[29610]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-09T08:25:37.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:37 smithi160 ceph-mon[34070]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-09T08:25:37.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:37 smithi188 ceph-mon[34645]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-09T08:25:39.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:39 smithi133 ceph-mon[29610]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T08:25:39.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]: dispatch 2024-05-09T08:25:39.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]': finished 2024-05-09T08:25:39.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:39 smithi133 ceph-mon[29610]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T08:25:39.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:25:39.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:39 smithi188 ceph-mon[34645]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T08:25:39.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]: dispatch 2024-05-09T08:25:39.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]': finished 2024-05-09T08:25:39.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:39 smithi188 ceph-mon[34645]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T08:25:39.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:25:40.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:39 smithi160 ceph-mon[34070]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T08:25:40.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]: dispatch 2024-05-09T08:25:40.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1515826066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af5f780d-dfa5-4680-a6cf-c821033f7540"}]': finished 2024-05-09T08:25:40.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:39 smithi160 ceph-mon[34070]: osdmap e65: 10 total, 9 up, 10 in 2024-05-09T08:25:40.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:25:41.147 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:40 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/766659666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:41.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:40 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/766659666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:41.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:40 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/766659666' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:25:42.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:41 smithi160 ceph-mon[34070]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T08:25:42.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:41 smithi133 ceph-mon[29610]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T08:25:42.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:41 smithi188 ceph-mon[34645]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-09T08:25:44.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:43 smithi133 ceph-mon[29610]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T08:25:44.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:43 smithi160 ceph-mon[34070]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T08:25:44.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:43 smithi188 ceph-mon[34645]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-09T08:25:45.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:45 smithi133 ceph-mon[29610]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:46.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:45 smithi160 ceph-mon[34070]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:46.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:45 smithi188 ceph-mon[34645]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:47.965 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:47 smithi160 ceph-mon[34070]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:48.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:47 smithi133 ceph-mon[29610]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:48.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:47 smithi188 ceph-mon[34645]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:50.064 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:49 smithi133 ceph-mon[29610]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:50.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T08:25:50.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:49 smithi160 ceph-mon[34070]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T08:25:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:49 smithi188 ceph-mon[34645]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T08:25:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:25:50.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:50 smithi188 ceph-mon[34645]: Deploying daemon osd.9 on smithi133 2024-05-09T08:25:51.110 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:50 smithi133 ceph-mon[29610]: Deploying daemon osd.9 on smithi133 2024-05-09T08:25:51.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:50 smithi160 ceph-mon[34070]: Deploying daemon osd.9 on smithi133 2024-05-09T08:25:51.965 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:51 smithi133 ceph-mon[29610]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:51 smithi160 ceph-mon[34070]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:51 smithi188 ceph-mon[34645]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:52 smithi160 ceph-mon[34070]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:53.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:52 smithi133 ceph-mon[29610]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:52 smithi188 ceph-mon[34645]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:55 smithi188 ceph-mon[34645]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:55 smithi160 ceph-mon[34070]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:55 smithi133 ceph-mon[29610]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:57 smithi188 ceph-mon[34645]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:57.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:57 smithi133 ceph-mon[29610]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:57.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:57.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:57.942 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:57 smithi160 ceph-mon[34070]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:25:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:25:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:25:59.998 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:25:59 smithi133 ceph-mon[29610]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:25:59 smithi160 ceph-mon[34070]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:00.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:25:59 smithi188 ceph-mon[34645]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:01.862 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:01 smithi133 ceph-mon[29610]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:01.862 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:01.862 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:01.862 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:01 smithi133 ceph-mon[29610]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T08:26:02.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:01 smithi160 ceph-mon[34070]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:02.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:02.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:02.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:01 smithi160 ceph-mon[34070]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T08:26:02.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:01 smithi188 ceph-mon[34645]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:02.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:02.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:02.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:01 smithi188 ceph-mon[34645]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T08:26:02.940 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 9 on host 'smithi133' 2024-05-09T08:26:02.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.939+0000 7f444aa00700 1 -- 172.21.15.133:0/3749451830 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f44540fd6d0 con 0x7f44380a2c20 2024-05-09T08:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 msgr2=0x7f44380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 0x7f44380a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f443c009c60 tx=0x7f443c009260 comp rx=0 tx=0).stop 2024-05-09T08:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 msgr2=0x7f445419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:02.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f444400fca0 tx=0x7f4444005250 comp rx=0 tx=0).stop 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 shutdown_connections 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4454108060 0x7f44541a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44540fffe0 0x7f445419a820 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f44380a2c20 0x7f44380a50e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 --2- 172.21.15.133:0/3749451830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44541009d0 0x7f445419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:02.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 >> 172.21.15.133:0/3749451830 conn(0x7f44540fbb70 msgr2=0x7f44540fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.942+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 shutdown_connections 2024-05-09T08:26:02.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:02.943+0000 7f4459200700 1 -- 172.21.15.133:0/3749451830 wait complete. 2024-05-09T08:26:03.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T08:26:03.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T08:26:03.650 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:03.699 DEBUG:teuthology.orchestra.run.smithi133:osd.9> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.9.service 2024-05-09T08:26:03.796 INFO:tasks.cephadm:Deploying osd.10 on smithi160 with /dev/vg_nvme/lv_1... 2024-05-09T08:26:03.796 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:03.823 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:03.824 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:03.824 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.824 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.824 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.824 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:03.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T08:26:03.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: osdmap e66: 10 total, 9 up, 10 in 2024-05-09T08:26:03.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-05-09T08:26:03.935 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:03.936 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:04.042 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:26:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:04 smithi160 ceph-mon[34070]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:26:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:04 smithi160 ceph-mon[34070]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T08:26:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:04.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:04 smithi133 ceph-mon[29610]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:26:04.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:04 smithi133 ceph-mon[29610]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T08:26:04.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:04.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:04.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:04 smithi188 ceph-mon[34645]: from='osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-05-09T08:26:04.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:04 smithi188 ceph-mon[34645]: osdmap e67: 10 total, 9 up, 10 in 2024-05-09T08:26:04.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:04.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:05.469 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] boot 2024-05-09T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:05.470 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:05 smithi160 ceph-mon[34070]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] boot 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:05.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:05 smithi133 ceph-mon[29610]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: osd.9 [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] boot 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: osdmap e68: 10 total, 10 up, 10 in 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T08:26:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:05 smithi188 ceph-mon[34645]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113672 s, 922 MB/s 2024-05-09T08:26:06.203 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2024-05-09T08:26:06.843 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:06 smithi160 ceph-mon[34070]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T08:26:06.872 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2024-05-09T08:26:06.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:06 smithi133 ceph-mon[29610]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T08:26:06.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:06 smithi188 ceph-mon[34645]: osdmap e69: 10 total, 10 up, 10 in 2024-05-09T08:26:07.168 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:26:07.589 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:07 smithi160 ceph-mon[34070]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:07.743 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:07 smithi133 ceph-mon[29610]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:07.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:07 smithi188 ceph-mon[34645]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- 172.21.15.160:0/1843382545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 msgr2=0x7f8e4c10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/1843382545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c10a7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400b0d0 tx=0x7f8e4400b3e0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- 172.21.15.160:0/1843382545 shutdown_connections 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/1843382545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c10a7d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/1843382545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c102380 0x7f8e4c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/1843382545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c103580 0x7f8e4c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- 172.21.15.160:0/1843382545 >> 172.21.15.160:0/1843382545 conn(0x7f8e4c0fd920 msgr2=0x7f8e4c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- 172.21.15.160:0/1843382545 shutdown_connections 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- 172.21.15.160:0/1843382545 wait complete. 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 Processor -- start 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.832+0000 7f8e50ff4700 1 -- start start 2024-05-09T08:26:08.835 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 0x7f8e4c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 0x7f8e4c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10dfe0 con 0x7f8e4c102380 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10de60 con 0x7f8e4c103f20 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c10e160 con 0x7f8e4c103580 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53390/0 (socket says 172.21.15.160:53390) 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 -- 172.21.15.160:0/617126922 learned_addr learned my addr 172.21.15.160:0/617126922 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e49d9b700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 0x7f8e4c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4a59c700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 0x7f8e4c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 -- 172.21.15.160:0/617126922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 msgr2=0x7f8e4c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:08.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 0x7f8e4c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 -- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 msgr2=0x7f8e4c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 0x7f8e4c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 -- 172.21.15.160:0/617126922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1a0160 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e49d9b700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 0x7f8e4c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4a59c700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 0x7f8e4c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e4ad9d700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e440062a0 tx=0x7f8e44007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44003c00 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1a03f0 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e44003da0 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.833+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005560 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.834+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a0950 con 0x7f8e4c103f20 2024-05-09T08:26:08.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.834+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c069010 con 0x7f8e4c103f20 2024-05-09T08:26:08.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.836+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8e44005700 con 0x7f8e4c103f20 2024-05-09T08:26:08.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.836+0000 7f8e3b7fe700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 0x7f8e340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:08.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.836+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f8e440d0060 con 0x7f8e4c103f20 2024-05-09T08:26:08.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.837+0000 7f8e4a59c700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 0x7f8e340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:08.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.837+0000 7f8e4a59c700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 0x7f8e340a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c1033e0 tx=0x7f8e3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:08.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:08.840+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e44009cb0 con 0x7f8e4c103f20 2024-05-09T08:26:09.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:09.046+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8e4c062820 con 0x7f8e340a2c20 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi133 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 4272M 2024-05-09T08:26:09.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:09.930 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi133 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 4272M 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:09.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi133 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 4272M 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:09.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:09.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:09.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:09.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:09.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:10 smithi160 ceph-mon[34070]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:10 smithi133 ceph-mon[29610]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:10.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:10 smithi188 ceph-mon[34645]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:11.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:11 smithi133 ceph-mon[29610]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:11.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:11 smithi188 ceph-mon[34645]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:12.146 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:11 smithi160 ceph-mon[34070]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/2434211259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]': finished 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:12.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:12 smithi188 ceph-mon[34645]: from='client.? 172.21.15.160:0/2522754706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/2434211259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]': finished 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:13.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:12 smithi160 ceph-mon[34070]: from='client.? 172.21.15.160:0/2522754706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/2434211259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]: dispatch 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83143e78-aa23-47d6-836a-3fd438f33c0d"}]': finished 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: osdmap e70: 11 total, 10 up, 11 in 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:13.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:12 smithi133 ceph-mon[29610]: from='client.? 172.21.15.160:0/2522754706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:13 smithi188 ceph-mon[34645]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:14.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:13 smithi160 ceph-mon[34070]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:14.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:13 smithi133 ceph-mon[29610]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:16.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:15 smithi160 ceph-mon[34070]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:16.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:15 smithi133 ceph-mon[29610]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:16.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:15 smithi188 ceph-mon[34645]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:17.952 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:17 smithi160 ceph-mon[34070]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:17.980 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:17 smithi133 ceph-mon[29610]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:18.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:17 smithi188 ceph-mon[34645]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:19.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:19 smithi160 ceph-mon[34070]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:19.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T08:26:19.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:19.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:19 smithi133 ceph-mon[29610]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:19.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T08:26:19.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:20.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:19 smithi188 ceph-mon[34645]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:20.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T08:26:20.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:20.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:20 smithi160 ceph-mon[34070]: Deploying daemon osd.10 on smithi160 2024-05-09T08:26:21.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:20 smithi133 ceph-mon[29610]: Deploying daemon osd.10 on smithi160 2024-05-09T08:26:21.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:20 smithi188 ceph-mon[34645]: Deploying daemon osd.10 on smithi160 2024-05-09T08:26:21.872 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:21 smithi160 ceph-mon[34070]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:22.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:21 smithi133 ceph-mon[29610]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:22.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:21 smithi188 ceph-mon[34645]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:23.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:23 smithi160 ceph-mon[34070]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:24.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:23 smithi133 ceph-mon[29610]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:24.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:23 smithi188 ceph-mon[34645]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:26.073 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:25 smithi160 ceph-mon[34070]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:26.074 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.074 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.074 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:26.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:25 smithi133 ceph-mon[29610]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:26.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:26.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:25 smithi188 ceph-mon[34645]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:26.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:26.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:27.085 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:26 smithi160 ceph-mon[34070]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:27.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:26 smithi133 ceph-mon[29610]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:27.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:26 smithi188 ceph-mon[34645]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:29.432 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 10 on host 'smithi160' 2024-05-09T08:26:29.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.431+0000 7f8e3b7fe700 1 -- 172.21.15.160:0/617126922 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8e4c062820 con 0x7f8e340a2c20 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 msgr2=0x7f8e340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 0x7f8e340a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c1033e0 tx=0x7f8e3c008040 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 msgr2=0x7f8e4c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8e440062a0 tx=0x7f8e44007e40 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 shutdown_connections 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e4c103f20 0x7f8e4c19fac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e4c102380 0x7f8e4c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8e340a2c20 0x7f8e340a50e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 --2- 172.21.15.160:0/617126922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e4c103580 0x7f8e4c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:29.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 >> 172.21.15.160:0/617126922 conn(0x7f8e4c0fd920 msgr2=0x7f8e4c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:29.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 shutdown_connections 2024-05-09T08:26:29.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T08:26:29.433+0000 7f8e50ff4700 1 -- 172.21.15.160:0/617126922 wait complete. 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:29.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T08:26:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:29.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:30.040 DEBUG:teuthology.orchestra.run.smithi160:osd.10> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.10.service 2024-05-09T08:26:30.042 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-05-09T08:26:30.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T08:26:30.286 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:26:30.659 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:30 smithi188 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T08:26:30.659 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:30 smithi188 ceph-mon[34645]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:30.659 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:30 smithi188 ceph-mon[34645]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T08:26:30.659 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:30.659 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:30 smithi188 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:30.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:30 smithi160 ceph-mon[34070]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T08:26:30.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:30 smithi160 ceph-mon[34070]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:30.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:30 smithi160 ceph-mon[34070]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T08:26:30.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:30.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:30 smithi160 ceph-mon[34070]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:30.663 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:26:30 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:26:30.570+0000 7f63bd752700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T08:26:30.663 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:26:30 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:26:30.582+0000 7f63b9545700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:26:30.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:30 smithi133 ceph-mon[29610]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T08:26:30.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:30 smithi133 ceph-mon[29610]: from='osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:30.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:30 smithi133 ceph-mon[29610]: osdmap e71: 11 total, 10 up, 11 in 2024-05-09T08:26:30.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:30.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:30 smithi133 ceph-mon[29610]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2024-05-09T08:26:31.717 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:26:31.717 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T08:26:31.717 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:31.717 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:31.717 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] boot 2024-05-09T08:26:31.718 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T08:26:31.718 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] boot 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T08:26:31.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: osdmap e72: 11 total, 10 up, 11 in 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: osd.10 [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] boot 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: osdmap e73: 11 total, 11 up, 11 in 2024-05-09T08:26:31.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114162 s, 918 MB/s 2024-05-09T08:26:32.433 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-05-09T08:26:33.091 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:32 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:26:33.092 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:32 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:26:33.092 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:32 smithi188 ceph-mon[34645]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T08:26:33.092 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:32 smithi188 ceph-mon[34645]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:33.115 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:32 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:26:33.115 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:32 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:26:33.115 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:32 smithi160 ceph-mon[34070]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T08:26:33.115 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:32 smithi160 ceph-mon[34070]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:33.120 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-05-09T08:26:33.377 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:26:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:32 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:26:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:32 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:26:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:32 smithi133 ceph-mon[29610]: osdmap e74: 11 total, 11 up, 11 in 2024-05-09T08:26:33.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:32 smithi133 ceph-mon[29610]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:34.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T08:26:34.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi160 2024-05-09T08:26:34.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:34.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T08:26:34.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi160 2024-05-09T08:26:34.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:34.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: osdmap e75: 11 total, 11 up, 11 in 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi160 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 4272M 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:34.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 -- 172.21.15.188:0/2973615531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1154103710 msgr2=0x7f1154103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 --2- 172.21.15.188:0/2973615531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1154103710 0x7f1154103b70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1148009990 tx=0x7f1148009ca0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 -- 172.21.15.188:0/2973615531 shutdown_connections 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 --2- 172.21.15.188:0/2973615531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11541040b0 0x7f115410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 --2- 172.21.15.188:0/2973615531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1154100690 0x7f1154100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 --2- 172.21.15.188:0/2973615531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1154103710 0x7f1154103b70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 -- 172.21.15.188:0/2973615531 >> 172.21.15.188:0/2973615531 conn(0x7f11540fbfd0 msgr2=0x7f11540fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 -- 172.21.15.188:0/2973615531 shutdown_connections 2024-05-09T08:26:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 -- 172.21.15.188:0/2973615531 wait complete. 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.073+0000 7f1158be8700 1 Processor -- start 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 -- start start 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1154100690 0x7f11541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1154103710 0x7f1154198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f115410e090 con 0x7f1154103710 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f115410df10 con 0x7f1154100690 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1158be8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f115410e210 con 0x7f11541040b0 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:40702/0 (socket says 172.21.15.188:40702) 2024-05-09T08:26:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 -- 172.21.15.188:0/1877328439 learned_addr learned my addr 172.21.15.188:0/1877328439 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:26:35.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f115259c700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1154100690 0x7f11541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:35.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 -- 172.21.15.188:0/1877328439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1154100690 msgr2=0x7f11541986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:35.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1154100690 0x7f11541986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 -- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1154103710 msgr2=0x7f1154198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1154103710 0x7f1154198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 -- 172.21.15.188:0/1877328439 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11541a0330 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.074+0000 7f1152d9d700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b9a0 tx=0x7f114c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.075+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c019070 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.075+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11541a0620 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.075+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c00ce10 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.075+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0073d0 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.075+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a0be0 con 0x7f11541040b0 2024-05-09T08:26:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.076+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1154069010 con 0x7f11541040b0 2024-05-09T08:26:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.077+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f114c01c070 con 0x7f11541040b0 2024-05-09T08:26:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.077+0000 7f11437fe700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 0x7f113c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.077+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7f114c0cdd90 con 0x7f11541040b0 2024-05-09T08:26:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.078+0000 7f115259c700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 0x7f113c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.079+0000 7f115259c700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 0x7f113c0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f115406a7d0 tx=0x7f1144009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:35.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.082+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f114c09b340 con 0x7f11541040b0 2024-05-09T08:26:35.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:34 smithi188 ceph-mon[34645]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T08:26:35.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:34 smithi188 ceph-mon[34645]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:26:35.287 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:35.286+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1154062760 con 0x7f113c0a2c20 2024-05-09T08:26:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:34 smithi160 ceph-mon[34070]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T08:26:35.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:34 smithi160 ceph-mon[34070]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:26:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:34 smithi133 ceph-mon[29610]: osdmap e76: 11 total, 11 up, 11 in 2024-05-09T08:26:35.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:34 smithi133 ceph-mon[29610]: pgmap v265: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-09T08:26:36.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:35 smithi188 ceph-mon[34645]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T08:26:36.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:35 smithi188 ceph-mon[34645]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:36.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:36.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:36.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:36.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:35 smithi160 ceph-mon[34070]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T08:26:36.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:35 smithi160 ceph-mon[34070]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:36.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:36.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:36.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:35 smithi133 ceph-mon[29610]: osdmap e77: 11 total, 11 up, 11 in 2024-05-09T08:26:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:35 smithi133 ceph-mon[29610]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:26:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T08:26:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T08:26:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:37.177 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:36 smithi188 ceph-mon[34645]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:26:37.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:36 smithi160 ceph-mon[34070]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:26:37.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:36 smithi133 ceph-mon[29610]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-05-09T08:26:38.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:38 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3957596909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:38 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:38 smithi188 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]': finished 2024-05-09T08:26:38.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:38 smithi188 ceph-mon[34645]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T08:26:38.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:38 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3957596909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:38 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:38 smithi160 ceph-mon[34070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]': finished 2024-05-09T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:38 smithi160 ceph-mon[34070]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T08:26:38.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:38.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:38 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3957596909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:38 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]: dispatch 2024-05-09T08:26:38.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:38 smithi133 ceph-mon[29610]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d934675-6b21-4c54-ac51-12593da6e97e"}]': finished 2024-05-09T08:26:38.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:38 smithi133 ceph-mon[29610]: osdmap e78: 12 total, 11 up, 12 in 2024-05-09T08:26:38.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:39.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:39 smithi160 ceph-mon[34070]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:26:39.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.188:0/3016954476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:39.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:39 smithi133 ceph-mon[29610]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:26:39.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.188:0/3016954476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:39.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:39 smithi188 ceph-mon[34645]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-09T08:26:39.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.188:0/3016954476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-09T08:26:41.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:41 smithi188 ceph-mon[34645]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:26:42.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:41 smithi160 ceph-mon[34070]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:26:42.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:41 smithi133 ceph-mon[29610]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-09T08:26:43.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:43 smithi188 ceph-mon[34645]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:44.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:43 smithi160 ceph-mon[34070]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:44.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:43 smithi133 ceph-mon[29610]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:45.982 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:45 smithi188 ceph-mon[34645]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:45.982 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T08:26:45.982 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:45 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:46.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:45 smithi160 ceph-mon[34070]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:46.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T08:26:46.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:45 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:46.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:45 smithi133 ceph-mon[29610]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:46.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T08:26:46.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:45 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:46.995 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:46 smithi188 ceph-mon[34645]: Deploying daemon osd.11 on smithi188 2024-05-09T08:26:47.063 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:46 smithi160 ceph-mon[34070]: Deploying daemon osd.11 on smithi188 2024-05-09T08:26:47.079 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:46 smithi133 ceph-mon[29610]: Deploying daemon osd.11 on smithi188 2024-05-09T08:26:47.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:47 smithi188 ceph-mon[34645]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:48.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:47 smithi160 ceph-mon[34070]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:48.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:47 smithi133 ceph-mon[29610]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:50.154 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:49 smithi188 ceph-mon[34645]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:49 smithi160 ceph-mon[34070]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:49 smithi133 ceph-mon[29610]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:51.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:51 smithi188 ceph-mon[34645]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:51.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:51.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:51.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:51 smithi160 ceph-mon[34070]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:51 smithi133 ceph-mon[29610]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:52 smithi188 ceph-mon[34645]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:53.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:52 smithi160 ceph-mon[34070]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:53.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:52 smithi133 ceph-mon[29610]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:54.674 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-05-09T08:26:54.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.674+0000 7f11437fe700 1 -- 172.21.15.188:0/1877328439 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1154062760 con 0x7f113c0a2c20 2024-05-09T08:26:54.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 msgr2=0x7f113c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:54.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 0x7f113c0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f115406a7d0 tx=0x7f1144009190 comp rx=0 tx=0).stop 2024-05-09T08:26:54.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 msgr2=0x7f115419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:54.677 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b9a0 tx=0x7f114c00bcb0 comp rx=0 tx=0).stop 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 shutdown_connections 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f113c0a2c20 0x7f113c0a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11541040b0 0x7f115419fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1154100690 0x7f11541986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 --2- 172.21.15.188:0/1877328439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1154103710 0x7f1154198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 >> 172.21.15.188:0/1877328439 conn(0x7f11540fbfd0 msgr2=0x7f1154106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 shutdown_connections 2024-05-09T08:26:54.678 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:26:54.677+0000 7f1158be8700 1 -- 172.21.15.188:0/1877328439 wait complete. 2024-05-09T08:26:55.371 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.11.service 2024-05-09T08:26:55.374 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T08:26:55.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-09T08:26:55.817 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:55.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:56.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:56 smithi160 ceph-mon[34070]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T08:26:56.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:56 smithi160 ceph-mon[34070]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:56.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:56 smithi160 ceph-mon[34070]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T08:26:56.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:56.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:56 smithi160 ceph-mon[34070]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:56.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:56 smithi133 ceph-mon[29610]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T08:26:56.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:56 smithi133 ceph-mon[29610]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:56.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:56 smithi133 ceph-mon[29610]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T08:26:56.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:56.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:56 smithi133 ceph-mon[29610]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:56.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:56 smithi188 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T08:26:56.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:56 smithi188 ceph-mon[34645]: from='osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:56.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:56 smithi188 ceph-mon[34645]: osdmap e79: 12 total, 11 up, 12 in 2024-05-09T08:26:56.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:56.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:56 smithi188 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-05-09T08:26:57.358 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:26:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:26:57.068+0000 7f4608f38700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T08:26:57.359 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:26:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:26:57.085+0000 7f4600d23700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:57 smithi160 ceph-mon[34070]: from='osd.11 ' entity='osd.11' 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:57 smithi133 ceph-mon[29610]: from='osd.11 ' entity='osd.11' 2024-05-09T08:26:57.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-05-09T08:26:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: osdmap e80: 12 total, 11 up, 12 in 2024-05-09T08:26:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:57.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:57 smithi188 ceph-mon[34645]: from='osd.11 ' entity='osd.11' 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.119+0000 7f68a1e00700 1 -- 172.21.15.133:0/3845607223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 msgr2=0x7f689c105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.119+0000 7f68a1e00700 1 --2- 172.21.15.133:0/3845607223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c105b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6888009990 tx=0x7f6888009ca0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.119+0000 7f68a1e00700 1 -- 172.21.15.133:0/3845607223 shutdown_connections 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.119+0000 7f68a1e00700 1 --2- 172.21.15.133:0/3845607223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c106060 0x7f689c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.120+0000 7f68a1e00700 1 --2- 172.21.15.133:0/3845607223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c1044c0 0x7f689c1048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.120+0000 7f68a1e00700 1 --2- 172.21.15.133:0/3845607223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c105b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.120+0000 7f68a1e00700 1 -- 172.21.15.133:0/3845607223 >> 172.21.15.133:0/3845607223 conn(0x7f689c0ffa60 msgr2=0x7f689c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.120+0000 7f68a1e00700 1 -- 172.21.15.133:0/3845607223 shutdown_connections 2024-05-09T08:26:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.120+0000 7f68a1e00700 1 -- 172.21.15.133:0/3845607223 wait complete. 2024-05-09T08:26:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 Processor -- start 2024-05-09T08:26:58.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 -- start start 2024-05-09T08:26:58.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 0x7f689c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:58.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 0x7f689c1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f689c110100 con 0x7f689c106060 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.121+0000 7f68a1e00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f689c10ff80 con 0x7f689c1044c0 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f68a1e00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f689c110280 con 0x7f689c1056c0 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:54196/0 (socket says 172.21.15.133:54196) 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f68a0a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 0x7f689c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 -- 172.21.15.133:0/1510106488 learned_addr learned my addr 172.21.15.133:0/1510106488 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f68a1400700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 0x7f689c1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 -- 172.21.15.133:0/1510106488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 msgr2=0x7f689c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 0x7f689c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 -- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 msgr2=0x7f689c1a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:58.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 0x7f689c1a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 -- 172.21.15.133:0/1510106488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c1a2310 con 0x7f689c1056c0 2024-05-09T08:26:58.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f68a1400700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 0x7f689c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:26:58.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f68a0a00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 0x7f689c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:26:58.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.122+0000 7f689be00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f688800f690 tx=0x7f688800f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.123+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888026070 con 0x7f689c1056c0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.123+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68880218f0 con 0x7f689c1056c0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.123+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1a25a0 con 0x7f689c1056c0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.123+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888017650 con 0x7f689c1056c0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.123+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1a2a50 con 0x7f689c1056c0 2024-05-09T08:26:58.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.124+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c069010 con 0x7f689c1056c0 2024-05-09T08:26:58.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.126+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6888029070 con 0x7f689c1056c0 2024-05-09T08:26:58.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.127+0000 7f6899600700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 0x7f68840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:26:58.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.127+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f68880cfaa0 con 0x7f689c1056c0 2024-05-09T08:26:58.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.127+0000 7f68a0a00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 0x7f68840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:26:58.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.128+0000 7f68a0a00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 0x7f68840a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f689c105520 tx=0x7f6890009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:26:58.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.131+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f688805d740 con 0x7f689c1056c0 2024-05-09T08:26:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.356+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f689c062f30 con 0x7f689c1056c0 2024-05-09T08:26:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.357+0000 7f6899600700 1 -- 172.21.15.133:0/1510106488 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f6888013070 con 0x7f689c1056c0 2024-05-09T08:26:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:26:58.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 msgr2=0x7f68840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 0x7f68840a5160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f689c105520 tx=0x7f6890009260 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 msgr2=0x7f689c19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f688800f690 tx=0x7f688800f6c0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 shutdown_connections 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f689c106060 0x7f689c1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f689c1044c0 0x7f689c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f68840a2ca0 0x7f68840a5160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 --2- 172.21.15.133:0/1510106488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f689c1056c0 0x7f689c19abf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.360+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 >> 172.21.15.133:0/1510106488 conn(0x7f689c0ffa60 msgr2=0x7f689c108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:26:58.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.361+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 shutdown_connections 2024-05-09T08:26:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:26:58.361+0000 7f68a1e00700 1 -- 172.21.15.133:0/1510106488 wait complete. 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: purged_snaps scrub starts 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: purged_snaps scrub ok 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] boot 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T08:26:58.783 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.784 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:58 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1510106488' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: purged_snaps scrub starts 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: purged_snaps scrub ok 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] boot 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:58 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1510106488' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T08:26:58.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: purged_snaps scrub starts 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: purged_snaps scrub ok 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: osd.11 [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] boot 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T08:26:58.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:58 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1510106488' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T08:26:59.149 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1715243218,"num_in_osds":12,"osd_in_since":1715243198,"num_remapped_pgs":0} 2024-05-09T08:26:59.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T08:26:59.583 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:26:59 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:59.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:59.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.919 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:26:59 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:26:59.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:26:59.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:26:59.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:26:59.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:26:59 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: Detected new or changed devices on smithi188 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:00.917 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:00 smithi133 ceph-mon[29610]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: Detected new or changed devices on smithi188 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:00.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:00 smithi188 ceph-mon[34645]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: Detected new or changed devices on smithi188 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 4272M 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:01.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:00 smithi160 ceph-mon[34070]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 -- 172.21.15.133:0/3740160220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c10a610 msgr2=0x7ffb8c10a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 --2- 172.21.15.133:0/3740160220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c10a610 0x7ffb8c10a9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0099c0 tx=0x7ffb7c009cd0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 -- 172.21.15.133:0/3740160220 shutdown_connections 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 --2- 172.21.15.133:0/3740160220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 --2- 172.21.15.133:0/3740160220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c10a610 0x7ffb8c10a9f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 --2- 172.21.15.133:0/3740160220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 -- 172.21.15.133:0/3740160220 >> 172.21.15.133:0/3740160220 conn(0x7ffb8c100030 msgr2=0x7ffb8c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 -- 172.21.15.133:0/3740160220 shutdown_connections 2024-05-09T08:27:01.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.754+0000 7ffb91600700 1 -- 172.21.15.133:0/3740160220 wait complete. 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.755+0000 7ffb91600700 1 Processor -- start 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.755+0000 7ffb91600700 1 -- start start 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c10a610 0x7ffb8c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1148d0 con 0x7ffb8c104f20 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c114750 con 0x7ffb8c104560 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb91600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c114a50 con 0x7ffb8c10a610 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49922/0 (socket says 172.21.15.133:49922) 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8a000700 1 -- 172.21.15.133:0/154028415 learned_addr learned my addr 172.21.15.133:0/154028415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8b400700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c10a610 0x7ffb8c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 -- 172.21.15.133:0/154028415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c10a610 msgr2=0x7ffb8c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c10a610 0x7ffb8c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 -- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 msgr2=0x7ffb8c1a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.756+0000 7ffb8aa00700 1 -- 172.21.15.133:0/154028415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c19b210 con 0x7ffb8c104560 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb8a000700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:01.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb8aa00700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1a08f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098e0 tx=0x7ffb7c003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c009e60 con 0x7ffb8c104560 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb7c003680 con 0x7ffb8c104560 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c19b4a0 con 0x7ffb8c104560 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00fd40 con 0x7ffb8c104560 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.757+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c1a77c0 con 0x7ffb8c104560 2024-05-09T08:27:01.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.758+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c10caa0 con 0x7ffb8c104560 2024-05-09T08:27:01.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.760+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ffb7c00f460 con 0x7ffb8c104560 2024-05-09T08:27:01.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.762+0000 7ffb83400700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 0x7ffb700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:01.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.762+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffb7c0cfb90 con 0x7ffb8c104560 2024-05-09T08:27:01.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.762+0000 7ffb8a000700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 0x7ffb700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:01.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.762+0000 7ffb8a000700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 0x7ffb700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c109be0 tx=0x7ffb78007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.766+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb7c013070 con 0x7ffb8c104560 2024-05-09T08:27:01.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:01 smithi133 ceph-mon[29610]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:01.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:01 smithi188 ceph-mon[34645]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:01.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.991+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb8c19bfe0 con 0x7ffb8c104560 2024-05-09T08:27:01.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.993+0000 7ffb83400700 1 -- 172.21.15.133:0/154028415 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18524 (secure 0 0 0) 0x7ffb7c09cf80 con 0x7ffb8c104560 2024-05-09T08:27:01.994 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:01.994 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:00.078249+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 msgr2=0x7ffb700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 0x7ffb700a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c109be0 tx=0x7ffb78007480 comp rx=0 tx=0).stop 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 msgr2=0x7ffb8c1a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1a08f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0098e0 tx=0x7ffb7c003e00 comp rx=0 tx=0).stop 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 shutdown_connections 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ffb700a2c20 0x7ffb700a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c10a610 0x7ffb8c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8c104560 0x7ffb8c1a08f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 --2- 172.21.15.133:0/154028415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb8c104f20 0x7ffb8c1a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 >> 172.21.15.133:0/154028415 conn(0x7ffb8c100030 msgr2=0x7ffb8c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 shutdown_connections 2024-05-09T08:27:01.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:01.996+0000 7ffb91600700 1 -- 172.21.15.133:0/154028415 wait complete. 2024-05-09T08:27:02.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:01 smithi160 ceph-mon[34070]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:02.811 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:27:02.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T08:27:02.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:02 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/154028415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:02.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:02 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/154028415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:03.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:02 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/154028415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:03.242 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:27:03.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:03 smithi133 ceph-mon[29610]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:03.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:03 smithi188 ceph-mon[34645]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:04.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:03 smithi160 ceph-mon[34070]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 -- 172.21.15.133:0/3555171737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 msgr2=0x7f1eb8106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3555171737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb8106f80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009990 tx=0x7f1ea4009ca0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 -- 172.21.15.133:0/3555171737 shutdown_connections 2024-05-09T08:27:05.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3555171737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb8107660 0x7f1eb810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3555171737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb81025d0 0x7f1eb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3555171737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb8106f80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.463+0000 7f1ec0600700 1 -- 172.21.15.133:0/3555171737 >> 172.21.15.133:0/3555171737 conn(0x7f1eb8079480 msgr2=0x7f1eb8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.464+0000 7f1ec0600700 1 -- 172.21.15.133:0/3555171737 shutdown_connections 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.464+0000 7f1ec0600700 1 -- 172.21.15.133:0/3555171737 wait complete. 2024-05-09T08:27:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.465+0000 7f1ec0600700 1 Processor -- start 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.465+0000 7f1ec0600700 1 -- start start 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 0x7f1eb819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb8107660 0x7f1eb81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8112650 con 0x7f1eb8102f80 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81124d0 con 0x7f1eb81025d0 2024-05-09T08:27:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ec0600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb81127d0 con 0x7f1eb8107660 2024-05-09T08:27:05.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebd200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:05.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebd200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43052/0 (socket says 172.21.15.133:43052) 2024-05-09T08:27:05.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebd200700 1 -- 172.21.15.133:0/3985309771 learned_addr learned my addr 172.21.15.133:0/3985309771 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:05.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 0x7f1eb819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebe600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb8107660 0x7f1eb81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 -- 172.21.15.133:0/3985309771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb8107660 msgr2=0x7f1eb81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb8107660 0x7f1eb81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 -- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 msgr2=0x7f1eb819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.466+0000 7f1ebdc00700 1 -- 172.21.15.133:0/3985309771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb8199110 con 0x7f1eb81025d0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1ebdc00700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 0x7f1eb819e7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00e9f0 tx=0x7f1eac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00c9c0 con 0x7f1eb81025d0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac00cb60 con 0x7f1eb81025d0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac018680 con 0x7f1eb81025d0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8199400 con 0x7f1eb81025d0 2024-05-09T08:27:05.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.467+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81a5670 con 0x7f1eb81025d0 2024-05-09T08:27:05.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.468+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb8069010 con 0x7f1eb81025d0 2024-05-09T08:27:05.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.470+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1eac010430 con 0x7f1eb81025d0 2024-05-09T08:27:05.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.472+0000 7f1eb6000700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 0x7f1ea00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:05.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.472+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1eac0cf090 con 0x7f1eb81025d0 2024-05-09T08:27:05.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.472+0000 7f1ebd200700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 0x7f1ea00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:05.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.473+0000 7f1ebd200700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 0x7f1ea00a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009750 tx=0x7f1ea400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:05.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.474+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1eac09c3f0 con 0x7f1eb81025d0 2024-05-09T08:27:05.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.695+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1eb802db80 con 0x7f1eb81025d0 2024-05-09T08:27:05.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.696+0000 7f1eb6000700 1 -- 172.21.15.133:0/3985309771 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f1eac09bcf0 con 0x7f1eb81025d0 2024-05-09T08:27:05.696 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 msgr2=0x7f1ea00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 0x7f1ea00a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009750 tx=0x7f1ea400b280 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 msgr2=0x7f1eb819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 0x7f1eb819e7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00e9f0 tx=0x7f1eac00ed00 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 shutdown_connections 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ea00a2c20 0x7f1ea00a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1eb8107660 0x7f1eb81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1eb81025d0 0x7f1eb819e7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 --2- 172.21.15.133:0/3985309771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1eb8102f80 0x7f1eb819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.699+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 >> 172.21.15.133:0/3985309771 conn(0x7f1eb8079480 msgr2=0x7f1eb8101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.700+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 shutdown_connections 2024-05-09T08:27:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:05.700+0000 7f1ec0600700 1 -- 172.21.15.133:0/3985309771 wait complete. 2024-05-09T08:27:05.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:05 smithi188 ceph-mon[34645]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:06.084 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:05 smithi133 ceph-mon[29610]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:06.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:05 smithi160 ceph-mon[34070]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.a on smithi133 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.d on smithi133 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.g on smithi133 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.j on smithi133 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.b on smithi160 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.e on smithi160 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.h on smithi160 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.k on smithi160 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-05-09T08:27:06.520 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-05-09T08:27:06.521 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-05-09T08:27:06.771 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:27:06.892 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:06 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3985309771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:06 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3985309771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:06 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3985309771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:08.075 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:07 smithi188 ceph-mon[34645]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:08.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:07 smithi160 ceph-mon[34070]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:08.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:07 smithi133 ceph-mon[29610]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:08.418 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- 172.21.15.188:0/2632134359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654102380 msgr2=0x7f96541027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 --2- 172.21.15.188:0/2632134359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654102380 0x7f96541027a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9644009990 tx=0x7f9644009ca0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- 172.21.15.188:0/2632134359 shutdown_connections 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 --2- 172.21.15.188:0/2632134359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654103f20 0x7f965410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 --2- 172.21.15.188:0/2632134359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654102380 0x7f96541027a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 --2- 172.21.15.188:0/2632134359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9654103580 0x7f96541039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- 172.21.15.188:0/2632134359 >> 172.21.15.188:0/2632134359 conn(0x7f96540fd920 msgr2=0x7f96540ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- 172.21.15.188:0/2632134359 shutdown_connections 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- 172.21.15.188:0/2632134359 wait complete. 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 Processor -- start 2024-05-09T08:27:08.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.418+0000 7f9659e59700 1 -- start start 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 0x7f96541985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9654103f20 0x7f965419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f965410dfc0 con 0x7f9654103f20 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f965410de40 con 0x7f9654102380 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f9659e59700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f965410e140 con 0x7f9654103580 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44976/0 (socket says 172.21.15.188:44976) 2024-05-09T08:27:08.421 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 -- 172.21.15.188:0/2369909279 learned_addr learned my addr 172.21.15.188:0/2369909279 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f96537fe700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 0x7f96541985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 -- 172.21.15.188:0/2369909279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 msgr2=0x7f96541985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 0x7f96541985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 -- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9654103f20 msgr2=0x7f965419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9654103f20 0x7f965419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 -- 172.21.15.188:0/2369909279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96541a0200 con 0x7f9654103580 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f96537fe700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 0x7f96541985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f964affd700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b560 tx=0x7f963c00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.419+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c015020 con 0x7f9654103580 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.420+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96541a04f0 con 0x7f9654103580 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.420+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f963c011d10 con 0x7f9654103580 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.420+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c00f2e0 con 0x7f9654103580 2024-05-09T08:27:08.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.420+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a0ab0 con 0x7f9654103580 2024-05-09T08:27:08.423 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.420+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9654069010 con 0x7f9654103580 2024-05-09T08:27:08.423 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.422+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f963c018020 con 0x7f9654103580 2024-05-09T08:27:08.423 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.422+0000 7f96517fa700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 0x7f96400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:08.423 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.422+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f963c0ce940 con 0x7f9654103580 2024-05-09T08:27:08.424 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.423+0000 7f96537fe700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 0x7f96400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:08.424 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.424+0000 7f96537fe700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 0x7f96400a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96440039c0 tx=0x7f9644005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:08.426 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.426+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f963c01f020 con 0x7f9654103580 2024-05-09T08:27:08.630 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.629+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) -- 0x7f9654062860 con 0x7f96400a2c20 2024-05-09T08:27:08.647 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.647+0000 7f96517fa700 1 -- 172.21.15.188:0/2369909279 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9654062860 con 0x7f96400a2c20 2024-05-09T08:27:08.647 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-05-09T08:27:08.649 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 msgr2=0x7f96400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:08.649 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 0x7f96400a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96440039c0 tx=0x7f9644005fb0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 msgr2=0x7f9654198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b560 tx=0x7f963c00b920 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 shutdown_connections 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9654103f20 0x7f965419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9654102380 0x7f96541985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96400a2c20 0x7f96400a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 --2- 172.21.15.188:0/2369909279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9654103580 0x7f9654198ae0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 >> 172.21.15.188:0/2369909279 conn(0x7f96540fd920 msgr2=0x7f9654106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 shutdown_connections 2024-05-09T08:27:08.650 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T08:27:08.649+0000 7f9659e59700 1 -- 172.21.15.188:0/2369909279 wait complete. 2024-05-09T08:27:09.333 DEBUG:teuthology.orchestra.run.smithi133:mds.a> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.a.service 2024-05-09T08:27:09.338 DEBUG:teuthology.orchestra.run.smithi133:mds.d> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.d.service 2024-05-09T08:27:09.382 DEBUG:teuthology.orchestra.run.smithi133:mds.g> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.g.service 2024-05-09T08:27:09.428 DEBUG:teuthology.orchestra.run.smithi133:mds.j> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.j.service 2024-05-09T08:27:09.475 DEBUG:teuthology.orchestra.run.smithi160:mds.b> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.b.service 2024-05-09T08:27:09.478 DEBUG:teuthology.orchestra.run.smithi160:mds.e> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.e.service 2024-05-09T08:27:09.521 DEBUG:teuthology.orchestra.run.smithi160:mds.h> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.h.service 2024-05-09T08:27:09.564 DEBUG:teuthology.orchestra.run.smithi160:mds.k> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.k.service 2024-05-09T08:27:09.608 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.c.service 2024-05-09T08:27:09.611 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.f.service 2024-05-09T08:27:09.654 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.i.service 2024-05-09T08:27:09.697 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.l.service 2024-05-09T08:27:09.741 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T08:27:09.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:27:09.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:09 smithi160 ceph-mon[34070]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:27:09.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:09 smithi160 ceph-mon[34070]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-09T08:27:09.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:09 smithi160 ceph-mon[34070]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:09.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:09.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:09 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:09 smithi133 ceph-mon[29610]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:27:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:09 smithi133 ceph-mon[29610]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-09T08:27:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:09 smithi133 ceph-mon[29610]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:09 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:09.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:09 smithi188 ceph-mon[34645]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:27:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:09 smithi188 ceph-mon[34645]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-09T08:27:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:09 smithi188 ceph-mon[34645]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:09 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:09.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 -- 172.21.15.133:0/3665170662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674074290 msgr2=0x7fe674074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 --2- 172.21.15.133:0/3665170662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674074290 0x7fe674074710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe660009930 tx=0x7fe66002a420 comp rx=0 tx=0).stop 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 -- 172.21.15.133:0/3665170662 shutdown_connections 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 --2- 172.21.15.133:0/3665170662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe67410dca0 0x7fe674110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 --2- 172.21.15.133:0/3665170662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674074290 0x7fe674074710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 --2- 172.21.15.133:0/3665170662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe674075870 0x7fe674073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 -- 172.21.15.133:0/3665170662 >> 172.21.15.133:0/3665170662 conn(0x7fe6740fcc40 msgr2=0x7fe6740ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 -- 172.21.15.133:0/3665170662 shutdown_connections 2024-05-09T08:27:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.958+0000 7fe679e00640 1 -- 172.21.15.133:0/3665170662 wait complete. 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.959+0000 7fe679e00640 1 Processor -- start 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.959+0000 7fe679e00640 1 -- start start 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674075870 0x7fe67419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 0x7fe6741a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113990 con 0x7fe674075870 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113810 con 0x7fe67410dca0 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe674113b10 con 0x7fe674074290 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:43304/0 (socket says 172.21.15.133:43304) 2024-05-09T08:27:09.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 -- 172.21.15.133:0/3636714726 learned_addr learned my addr 172.21.15.133:0/3636714726 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe673e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674075870 0x7fe67419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679400640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 0x7fe6741a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 -- 172.21.15.133:0/3636714726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 msgr2=0x7fe6741a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 0x7fe6741a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 -- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674075870 msgr2=0x7fe67419e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674075870 0x7fe67419e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 -- 172.21.15.133:0/3636714726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741a5c20 con 0x7fe674074290 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe679400640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 0x7fe6741a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.960+0000 7fe678a00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9f0 tx=0x7fe66800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.961+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66800cd80 con 0x7fe674074290 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.961+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe668010910 con 0x7fe674074290 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.961+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741a5f10 con 0x7fe674074290 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.961+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680188f0 con 0x7fe674074290 2024-05-09T08:27:09.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.961+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741a6370 con 0x7fe674074290 2024-05-09T08:27:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.962+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe674103e70 con 0x7fe674074290 2024-05-09T08:27:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.964+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe668010ac0 con 0x7fe674074290 2024-05-09T08:27:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.965+0000 7fe671600640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 0x7fe65c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.965+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe6680cf0a0 con 0x7fe674074290 2024-05-09T08:27:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.965+0000 7fe673e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 0x7fe65c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:09.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.966+0000 7fe673e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 0x7fe65c0a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe660009900 tx=0x7fe6600315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:09.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:09.969+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe66809c400 con 0x7fe674074290 2024-05-09T08:27:10.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.180+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe674062500 con 0x7fe674074290 2024-05-09T08:27:10.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.181+0000 7fe671600640 1 -- 172.21.15.133:0/3636714726 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18524 (secure 0 0 0) 0x7fe66809bd00 con 0x7fe674074290 2024-05-09T08:27:10.182 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:10.182 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:00.078249+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:27:10.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 msgr2=0x7fe65c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 0x7fe65c0a5050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe660009900 tx=0x7fe6600315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 msgr2=0x7fe67419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9f0 tx=0x7fe66800eeb0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 shutdown_connections 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe65c0a2b90 0x7fe65c0a5050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe67410dca0 0x7fe6741a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe674075870 0x7fe67419e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 --2- 172.21.15.133:0/3636714726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674074290 0x7fe67419e000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.184+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 >> 172.21.15.133:0/3636714726 conn(0x7fe6740fcc40 msgr2=0x7fe674101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.185+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 shutdown_connections 2024-05-09T08:27:10.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.185+0000 7fe679e00640 1 -- 172.21.15.133:0/3636714726 wait complete. 2024-05-09T08:27:10.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:27:10.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:27:10.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 -- 172.21.15.133:0/825510915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 msgr2=0x7fea24073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 --2- 172.21.15.133:0/825510915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea24073cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 -- 172.21.15.133:0/825510915 shutdown_connections 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 --2- 172.21.15.133:0/825510915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 0x7fea2410bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 --2- 172.21.15.133:0/825510915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 --2- 172.21.15.133:0/825510915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea24073cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 -- 172.21.15.133:0/825510915 >> 172.21.15.133:0/825510915 conn(0x7fea240fce10 msgr2=0x7fea240ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 -- 172.21.15.133:0/825510915 shutdown_connections 2024-05-09T08:27:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.403+0000 7fea2b800640 1 -- 172.21.15.133:0/825510915 wait complete. 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.404+0000 7fea2b800640 1 Processor -- start 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.404+0000 7fea2b800640 1 -- start start 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea2419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 0x7fea241a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410f710 con 0x7fea24109740 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410f590 con 0x7fea24074290 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea2b800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea2410f890 con 0x7fea24075870 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea1be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea2419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:41086/0 (socket says 172.21.15.133:41086) 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 -- 172.21.15.133:0/270722692 learned_addr learned my addr 172.21.15.133:0/270722692 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea29800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 0x7fea241a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 -- 172.21.15.133:0/270722692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 msgr2=0x7fea2419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea2419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 -- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 msgr2=0x7fea241a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 0x7fea241a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea28e00640 1 -- 172.21.15.133:0/270722692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea241a19e0 con 0x7fea24074290 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.405+0000 7fea1be00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea2419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea28e00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea14031440 tx=0x7fea14032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1403f030 con 0x7fea24074290 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea14003ce0 con 0x7fea24074290 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea241a1c70 con 0x7fea24074290 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14035070 con 0x7fea24074290 2024-05-09T08:27:10.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.406+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a21b0 con 0x7fea24074290 2024-05-09T08:27:10.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.407+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24108470 con 0x7fea24074290 2024-05-09T08:27:10.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.409+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fea14042030 con 0x7fea24074290 2024-05-09T08:27:10.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.410+0000 7fea22000640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 0x7fea080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.411+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fea140ef0d0 con 0x7fea24074290 2024-05-09T08:27:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.411+0000 7fea1be00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 0x7fea080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.411+0000 7fea1be00640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 0x7fea080a50d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea2419b2d0 tx=0x7fea0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.415+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea140bbde0 con 0x7fea24074290 2024-05-09T08:27:10.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.617+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea24062430 con 0x7fea24074290 2024-05-09T08:27:10.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.618+0000 7fea22000640 1 -- 172.21.15.133:0/270722692 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fea140c00e0 con 0x7fea24074290 2024-05-09T08:27:10.618 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 msgr2=0x7fea080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 0x7fea080a50d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea2419b2d0 tx=0x7fea0c00b410 comp rx=0 tx=0).stop 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 msgr2=0x7fea24199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea14031440 tx=0x7fea14032ce0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 shutdown_connections 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea080a2c10 0x7fea080a50d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24109740 0x7fea241a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea24075870 0x7fea2419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 --2- 172.21.15.133:0/270722692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea24074290 0x7fea24199dc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 >> 172.21.15.133:0/270722692 conn(0x7fea240fce10 msgr2=0x7fea24105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:10.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 shutdown_connections 2024-05-09T08:27:10.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.621+0000 7fea2b800640 1 -- 172.21.15.133:0/270722692 wait complete. 2024-05-09T08:27:10.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3636714726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:10.841 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:10 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/270722692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 -- 172.21.15.133:0/1134369049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 msgr2=0x7ff9941100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 --2- 172.21.15.133:0/1134369049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941100f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009ca0 tx=0x7ff98c02a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 -- 172.21.15.133:0/1134369049 shutdown_connections 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 --2- 172.21.15.133:0/1134369049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941100f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 --2- 172.21.15.133:0/1134369049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff994109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 --2- 172.21.15.133:0/1134369049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff9941080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 -- 172.21.15.133:0/1134369049 >> 172.21.15.133:0/1134369049 conn(0x7ff994076c70 msgr2=0x7ff994079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 -- 172.21.15.133:0/1134369049 shutdown_connections 2024-05-09T08:27:10.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.855+0000 7ff999a00640 1 -- 172.21.15.133:0/1134369049 wait complete. 2024-05-09T08:27:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 Processor -- start 2024-05-09T08:27:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 -- start start 2024-05-09T08:27:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff99419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff99419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff994113930 con 0x7ff994109860 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff9941137b0 con 0x7ff994107cb0 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff994113ab0 con 0x7ff994108ea0 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43114/0 (socket says 172.21.15.133:43114) 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff999000640 1 -- 172.21.15.133:0/433072747 learned_addr learned my addr 172.21.15.133:0/433072747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.857+0000 7ff993e00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff99419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff98b400640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff99419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 -- 172.21.15.133:0/433072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 msgr2=0x7ff99419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff99419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 -- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 msgr2=0x7ff9941a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 -- 172.21.15.133:0/433072747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9941a5ba0 con 0x7ff994107cb0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff98b400640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff99419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff999000640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff993e00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff99419dfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e9c0 tx=0x7ff98400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.858+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400ccc0 con 0x7ff994107cb0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.859+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9840035f0 con 0x7ff994107cb0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.859+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9941a5e90 con 0x7ff994107cb0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.859+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400ce60 con 0x7ff994107cb0 2024-05-09T08:27:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.859+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9941a62f0 con 0x7ff994107cb0 2024-05-09T08:27:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.860+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99410efd0 con 0x7ff994107cb0 2024-05-09T08:27:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.862+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff9840037a0 con 0x7ff994107cb0 2024-05-09T08:27:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.863+0000 7ff991600640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 0x7ff9780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.863+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff9840ceff0 con 0x7ff994107cb0 2024-05-09T08:27:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.863+0000 7ff98b400640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 0x7ff9780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.864+0000 7ff98b400640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 0x7ff9780a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff99419f490 tx=0x7ff97c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:10.867+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff98409c3b0 con 0x7ff994107cb0 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3636714726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:10 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/270722692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3636714726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:10 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/270722692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:11.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.108+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff99402d1c0 con 0x7ff994107cb0 2024-05-09T08:27:11.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.108+0000 7ff991600640 1 -- 172.21.15.133:0/433072747 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff98409bcb0 con 0x7ff994107cb0 2024-05-09T08:27:11.109 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:11.109 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T08:27:11.110 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2024-05-09T08:27:11.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 msgr2=0x7ff9780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 0x7ff9780a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff99419f490 tx=0x7ff97c005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 msgr2=0x7ff99419dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff99419dfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e9c0 tx=0x7ff98400ee80 comp rx=0 tx=0).stop 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 shutdown_connections 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9780a2b90 0x7ff9780a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff994109860 0x7ff9941a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff994108ea0 0x7ff99419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:11.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 --2- 172.21.15.133:0/433072747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994107cb0 0x7ff99419dfd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:11.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.111+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 >> 172.21.15.133:0/433072747 conn(0x7ff994076c70 msgr2=0x7ff99410c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:11.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.112+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 shutdown_connections 2024-05-09T08:27:11.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:11.112+0000 7ff999a00640 1 -- 172.21.15.133:0/433072747 wait complete. 2024-05-09T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:11 smithi188 ceph-mon[34645]: Deploying daemon mds.l on smithi188 2024-05-09T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:11 smithi188 ceph-mon[34645]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:11 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/433072747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:11.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:11 smithi160 ceph-mon[34070]: Deploying daemon mds.l on smithi188 2024-05-09T08:27:11.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:11 smithi160 ceph-mon[34070]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:11.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:11 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/433072747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:11.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:11 smithi133 ceph-mon[29610]: Deploying daemon mds.l on smithi188 2024-05-09T08:27:11.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:11 smithi133 ceph-mon[29610]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:11.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:11 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/433072747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:12.183 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:11 smithi188 systemd[1]: Starting Ceph mds.l for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:12.517 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 podman[64870]: 2024-05-09 08:27:12.115580866 +0000 UTC m=+0.017710311 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:12.517 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 podman[64870]: 2024-05-09 08:27:12.259781743 +0000 UTC m=+0.161911221 container create acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T08:27:12.517 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 podman[64870]: 2024-05-09 08:27:12.518031974 +0000 UTC m=+0.420161411 container init acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T08:27:12.933 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 podman[64870]: 2024-05-09 08:27:12.520881948 +0000 UTC m=+0.423011398 container start acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T08:27:12.933 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 ceph-mds[64889]: -- 172.21.15.188:0/883386992 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0a9c91a40 con 0x55d0a9c65000 2024-05-09T08:27:12.933 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l[64885]: starting mds.l at 2024-05-09T08:27:12.933 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 bash[64870]: acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea 2024-05-09T08:27:12.933 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:27:12 smithi188 systemd[1]: Started Ceph mds.l for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:13.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.344+0000 7f9635e00640 1 -- 172.21.15.133:0/2626510701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630108ea0 msgr2=0x7f9630109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.344+0000 7f9635e00640 1 --2- 172.21.15.133:0/2626510701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630108ea0 0x7f9630109320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9618009900 tx=0x7f961802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 -- 172.21.15.133:0/2626510701 shutdown_connections 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 --2- 172.21.15.133:0/2626510701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630109860 0x7f96301100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 --2- 172.21.15.133:0/2626510701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630108ea0 0x7f9630109320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 --2- 172.21.15.133:0/2626510701 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630107cb0 0x7f96301080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 -- 172.21.15.133:0/2626510701 >> 172.21.15.133:0/2626510701 conn(0x7f9630076c70 msgr2=0x7f9630079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:13.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 -- 172.21.15.133:0/2626510701 shutdown_connections 2024-05-09T08:27:13.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.345+0000 7f9635e00640 1 -- 172.21.15.133:0/2626510701 wait complete. 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 Processor -- start 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 -- start start 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 0x7f963019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 0x7f963019e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630109860 0x7f96301a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9630113980 con 0x7f9630109860 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9630113800 con 0x7f9630108ea0 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.346+0000 7f9635e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9630113b00 con 0x7f9630107cb0 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9635400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630109860 0x7f96301a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9634a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 0x7f963019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 0x7f963019e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9634a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 0x7f963019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:43348/0 (socket says 172.21.15.133:43348) 2024-05-09T08:27:13.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9634a00640 1 -- 172.21.15.133:0/2027601764 learned_addr learned my addr 172.21.15.133:0/2027601764 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 -- 172.21.15.133:0/2027601764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 msgr2=0x7f963019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 0x7f963019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 -- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630109860 msgr2=0x7f96301a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630109860 0x7f96301a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 -- 172.21.15.133:0/2027601764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96301a5bf0 con 0x7f9630108ea0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.347+0000 7f9627e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 0x7f963019e560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f961802a910 tx=0x7f9618032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.348+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961803f030 con 0x7f9630108ea0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.348+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9618003ce0 con 0x7f9630108ea0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.348+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96301a5e80 con 0x7f9630108ea0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.348+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618035070 con 0x7f9630108ea0 2024-05-09T08:27:13.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.348+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96301a62f0 con 0x7f9630108ea0 2024-05-09T08:27:13.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.349+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963010ef70 con 0x7f9630108ea0 2024-05-09T08:27:13.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.351+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9618042030 con 0x7f9630108ea0 2024-05-09T08:27:13.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.352+0000 7f962e000640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 0x7f96140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:13.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.352+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f96180ef4b0 con 0x7f9630108ea0 2024-05-09T08:27:13.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.352+0000 7f9634a00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 0x7f96140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:13.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.353+0000 7f9634a00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 0x7f96140a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f96200058c0 tx=0x7f9620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:13.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.354+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96180bc200 con 0x7f9630108ea0 2024-05-09T08:27:13.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.597+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963002d1c0 con 0x7f9630108ea0 2024-05-09T08:27:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.598+0000 7f962e000640 1 -- 172.21.15.133:0/2027601764 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f96180c0500 con 0x7f9630108ea0 2024-05-09T08:27:13.598 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:13.598 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T08:27:13.598 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 2 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 msgr2=0x7f96140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 0x7f96140a50d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f96200058c0 tx=0x7f9620005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 msgr2=0x7f963019e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 0x7f963019e560 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f961802a910 tx=0x7f9618032ce0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 shutdown_connections 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f96140a2c10 0x7f96140a50d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9630109860 0x7f96301a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9630108ea0 0x7f963019e560 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 --2- 172.21.15.133:0/2027601764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9630107cb0 0x7f963019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.600+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 >> 172.21.15.133:0/2027601764 conn(0x7f9630076c70 msgr2=0x7f963010c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.601+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 shutdown_connections 2024-05-09T08:27:13.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:13.601+0000 7f9635e00640 1 -- 172.21.15.133:0/2027601764 wait complete. 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6832/1493326462,v1:172.21.15.188:6833/1493326462] up:boot 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: fsmap 1 up:standby 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:13.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:13.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6832/1493326462,v1:172.21.15.188:6833/1493326462] up:boot 2024-05-09T08:27:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: fsmap 1 up:standby 2024-05-09T08:27:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T08:27:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:13.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:13.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:13.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6832/1493326462,v1:172.21.15.188:6833/1493326462] up:boot 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: fsmap 1 up:standby 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:13.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:14.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:14 smithi160 ceph-mon[34070]: Deploying daemon mds.k on smithi160 2024-05-09T08:27:14.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:14 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2027601764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:14.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:14 smithi133 ceph-mon[29610]: Deploying daemon mds.k on smithi160 2024-05-09T08:27:14.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:14 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2027601764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:14.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:14 smithi188 ceph-mon[34645]: Deploying daemon mds.k on smithi160 2024-05-09T08:27:14.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:14 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2027601764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:15.163 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:14 smithi160 systemd[1]: Starting Ceph mds.k for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:15.424 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 podman[63500]: 2024-05-09 08:27:15.07462315 +0000 UTC m=+0.018511458 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:15.424 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 podman[63500]: 2024-05-09 08:27:15.196660465 +0000 UTC m=+0.140548758 container create 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T08:27:15.424 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 podman[63500]: 2024-05-09 08:27:15.424536981 +0000 UTC m=+0.368425273 container init 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:27:15.622 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:15.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.818+0000 7f2013200640 1 -- 172.21.15.133:0/2384417889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c075870 msgr2=0x7f200c073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.818+0000 7f2013200640 1 --2- 172.21.15.133:0/2384417889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c075870 0x7f200c073cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009900 tx=0x7f1ffc02a410 comp rx=0 tx=0).stop 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 -- 172.21.15.133:0/2384417889 shutdown_connections 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 --2- 172.21.15.133:0/2384417889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c109740 0x7f200c10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 --2- 172.21.15.133:0/2384417889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c074290 0x7f200c074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 --2- 172.21.15.133:0/2384417889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c075870 0x7f200c073cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 -- 172.21.15.133:0/2384417889 >> 172.21.15.133:0/2384417889 conn(0x7f200c0fce10 msgr2=0x7f200c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 -- 172.21.15.133:0/2384417889 shutdown_connections 2024-05-09T08:27:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.819+0000 7f2013200640 1 -- 172.21.15.133:0/2384417889 wait complete. 2024-05-09T08:27:15.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 Processor -- start 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 -- start start 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c074290 0x7f200c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c075870 0x7f200c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f200c10f730 con 0x7f200c075870 2024-05-09T08:27:15.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f200c10f5b0 con 0x7f200c109740 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.820+0000 7f2013200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f200c10f8b0 con 0x7f200c074290 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f200b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c075870 0x7f200c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f200be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c074290 0x7f200c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:60302/0 (socket says 172.21.15.133:60302) 2024-05-09T08:27:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 -- 172.21.15.133:0/2048242914 learned_addr learned my addr 172.21.15.133:0/2048242914 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 -- 172.21.15.133:0/2048242914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c074290 msgr2=0x7f200c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c074290 0x7f200c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 -- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c075870 msgr2=0x7f200c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c075870 0x7f200c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 -- 172.21.15.133:0/2048242914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c1a19b0 con 0x7f200c109740 2024-05-09T08:27:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.821+0000 7f2011200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f200400eb10 tx=0x7f200400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.822+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004004b20 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.822+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2004004cc0 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.822+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004010760 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.822+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c1a1ca0 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.822+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c1a21c0 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.823+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c108470 con 0x7f200c109740 2024-05-09T08:27:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.825+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2004005020 con 0x7f200c109740 2024-05-09T08:27:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.826+0000 7f2008c00640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.826+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f20040cf200 con 0x7f200c109740 2024-05-09T08:27:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.826+0000 7f200be00640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.827+0000 7f200be00640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc031440 tx=0x7f1ffc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:15.830+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f200409c560 con 0x7f200c109740 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:15 smithi160 ceph-mon[34070]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 podman[63500]: 2024-05-09 08:27:15.427337088 +0000 UTC m=+0.371225384 container start 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 ceph-mds[63519]: -- 172.21.15.160:0/1156517829 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55979ee03a40 con 0x55979edd7000 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k[63515]: starting mds.k at 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 bash[63500]: 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 2024-05-09T08:27:15.913 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:27:15 smithi160 systemd[1]: Started Ceph mds.k for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:15.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:15 smithi133 ceph-mon[29610]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:15.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:15 smithi188 ceph-mon[34645]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.070+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f200c062430 con 0x7f200c109740 2024-05-09T08:27:16.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.071+0000 7f2008c00640 1 -- 172.21.15.133:0/2048242914 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f200409be60 con 0x7f200c109740 2024-05-09T08:27:16.071 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:16.071 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:27:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.074+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 msgr2=0x7f1ff00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.074+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc031440 tx=0x7f1ffc0315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.074+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 msgr2=0x7f200c1a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.074+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f200400eb10 tx=0x7f200400c2d0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 shutdown_connections 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f200c109740 0x7f200c1a1310 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f200c075870 0x7f200c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 --2- 172.21.15.133:0/2048242914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f200c074290 0x7f200c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 >> 172.21.15.133:0/2048242914 conn(0x7f200c0fce10 msgr2=0x7f200c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 shutdown_connections 2024-05-09T08:27:16.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:16.075+0000 7f2013200640 1 -- 172.21.15.133:0/2048242914 wait complete. 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6832/4109875508,v1:172.21.15.160:6833/4109875508] up:boot 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: fsmap 2 up:standby 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:16.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:16.858 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:16.858 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: Deploying daemon mds.j on smithi133 2024-05-09T08:27:16.858 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:16 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2048242914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6832/4109875508,v1:172.21.15.160:6833/4109875508] up:boot 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: fsmap 2 up:standby 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: Deploying daemon mds.j on smithi133 2024-05-09T08:27:16.902 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:16 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2048242914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6832/4109875508,v1:172.21.15.160:6833/4109875508] up:boot 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: fsmap 2 up:standby 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: Deploying daemon mds.j on smithi133 2024-05-09T08:27:16.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:16 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2048242914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:17.770 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:17 smithi133 ceph-mon[29610]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:17.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:17 smithi160 ceph-mon[34070]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:17.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:17 smithi188 ceph-mon[34645]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:18.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:18.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 -- 172.21.15.133:0/1972874883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d341100b0 msgr2=0x7f0d341124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 --2- 172.21.15.133:0/1972874883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d341100b0 0x7f0d341124a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009be0 tx=0x7f0d2c02a710 comp rx=0 tx=0).stop 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 -- 172.21.15.133:0/1972874883 shutdown_connections 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 --2- 172.21.15.133:0/1972874883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d341100b0 0x7f0d341124a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 --2- 172.21.15.133:0/1972874883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 0x7f0d3410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 --2- 172.21.15.133:0/1972874883 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d34108830 0x7f0d34108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.373+0000 7f0d3a200640 1 -- 172.21.15.133:0/1972874883 >> 172.21.15.133:0/1972874883 conn(0x7f0d3406e6d0 msgr2=0x7f0d34070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.375+0000 7f0d3a200640 1 -- 172.21.15.133:0/1972874883 shutdown_connections 2024-05-09T08:27:18.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.375+0000 7f0d3a200640 1 -- 172.21.15.133:0/1972874883 wait complete. 2024-05-09T08:27:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.376+0000 7f0d3a200640 1 Processor -- start 2024-05-09T08:27:18.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.376+0000 7f0d3a200640 1 -- start start 2024-05-09T08:27:18.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.377+0000 7f0d3a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.378+0000 7f0d3a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 0x7f0d3419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.378+0000 7f0d3a200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d341100b0 0x7f0d341a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.378+0000 7f0d3a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34113b10 con 0x7f0d34108830 2024-05-09T08:27:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.378+0000 7f0d3a200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34113990 con 0x7f0d341100b0 2024-05-09T08:27:18.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.378+0000 7f0d3a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34113c90 con 0x7f0d34109190 2024-05-09T08:27:18.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.379+0000 7f0d33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:18.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.379+0000 7f0d33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43440/0 (socket says 172.21.15.133:43440) 2024-05-09T08:27:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.379+0000 7f0d33400640 1 -- 172.21.15.133:0/709670830 learned_addr learned my addr 172.21.15.133:0/709670830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d32a00640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 0x7f0d3419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d33400640 1 -- 172.21.15.133:0/709670830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 msgr2=0x7f0d3419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d33400640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 0x7f0d3419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d33400640 1 -- 172.21.15.133:0/709670830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d341100b0 msgr2=0x7f0d341a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:18.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d33400640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d341100b0 0x7f0d341a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.380+0000 7f0d33400640 1 -- 172.21.15.133:0/709670830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d341a5d30 con 0x7f0d34108830 2024-05-09T08:27:18.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.381+0000 7f0d33400640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400ff50 tx=0x7f0d24015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:18.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.382+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24022070 con 0x7f0d34108830 2024-05-09T08:27:18.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.382+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d24013e30 con 0x7f0d34108830 2024-05-09T08:27:18.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.382+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d341a5fc0 con 0x7f0d34108830 2024-05-09T08:27:18.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.382+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2401cae0 con 0x7f0d34108830 2024-05-09T08:27:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.382+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d341a64b0 con 0x7f0d34108830 2024-05-09T08:27:18.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.384+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d140052c0 con 0x7f0d34108830 2024-05-09T08:27:18.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.389+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0d24025070 con 0x7f0d34108830 2024-05-09T08:27:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.392+0000 7f0d2be00640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 0x7f0d180a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.392+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0d240d6770 con 0x7f0d34108830 2024-05-09T08:27:18.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.392+0000 7f0d32a00640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 0x7f0d180a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:18.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.393+0000 7f0d32a00640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 0x7f0d180a51a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005fd0 tx=0x7f0d20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:18.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.398+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d24018070 con 0x7f0d34108830 2024-05-09T08:27:18.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.647+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d140055b0 con 0x7f0d34108830 2024-05-09T08:27:18.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.647+0000 7f0d2be00640 1 -- 172.21.15.133:0/709670830 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f0d240a3ad0 con 0x7f0d34108830 2024-05-09T08:27:18.647 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:18.647 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:27:18.648 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 msgr2=0x7f0d180a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 0x7f0d180a51a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005fd0 tx=0x7f0d20005850 comp rx=0 tx=0).stop 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 msgr2=0x7f0d3419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400ff50 tx=0x7f0d24015f90 comp rx=0 tx=0).stop 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 shutdown_connections 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0d180a2ce0 0x7f0d180a51a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d341100b0 0x7f0d341a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d34109190 0x7f0d3419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 --2- 172.21.15.133:0/709670830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d34108830 0x7f0d3419e110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:18.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 >> 172.21.15.133:0/709670830 conn(0x7f0d3406e6d0 msgr2=0x7f0d340702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:18.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 shutdown_connections 2024-05-09T08:27:18.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:18.650+0000 7f0d3a200640 1 -- 172.21.15.133:0/709670830 wait complete. 2024-05-09T08:27:18.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:18 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/709670830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:19.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:18 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/709670830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:19.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:18 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/709670830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:19.259 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 podman[69595]: 2024-05-09 08:27:19.006126335 +0000 UTC m=+0.164455413 container create c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T08:27:19.259 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 podman[69595]: 2024-05-09 08:27:19.258976441 +0000 UTC m=+0.417305492 container init c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-09T08:27:19.677 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 podman[69595]: 2024-05-09 08:27:19.266262532 +0000 UTC m=+0.424591573 container start c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T08:27:19.678 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mds[69620]: -- 172.21.15.133:0/3175857379 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e9d523a40 con 0x557e9d4f6800 2024-05-09T08:27:19.678 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j[69614]: starting mds.j at 2024-05-09T08:27:19.678 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 bash[69595]: c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff 2024-05-09T08:27:19.678 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:27:19 smithi133 systemd[1]: Started Ceph mds.j for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:20.039 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:20.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:20.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:20.806 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:20 smithi133 ceph-mon[29610]: Deploying daemon mds.d on smithi133 2024-05-09T08:27:20.807 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:20 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6834/1799489914,v1:172.21.15.133:6835/1799489914] up:boot 2024-05-09T08:27:20.807 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:20 smithi133 ceph-mon[29610]: fsmap 3 up:standby 2024-05-09T08:27:20.807 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:20 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.896+0000 7f3af9000640 1 -- 172.21.15.133:0/3017622638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 msgr2=0x7f3af410bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.896+0000 7f3af9000640 1 --2- 172.21.15.133:0/3017622638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af410bcf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009ca0 tx=0x7f3aec02a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 -- 172.21.15.133:0/3017622638 shutdown_connections 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 --2- 172.21.15.133:0/3017622638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af410bcf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 --2- 172.21.15.133:0/3017622638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af4104aa0 0x7f3af4104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 --2- 172.21.15.133:0/3017622638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af41038b0 0x7f3af4103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 -- 172.21.15.133:0/3017622638 >> 172.21.15.133:0/3017622638 conn(0x7f3af40fee40 msgr2=0x7f3af4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:20.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 -- 172.21.15.133:0/3017622638 shutdown_connections 2024-05-09T08:27:20.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.897+0000 7f3af9000640 1 -- 172.21.15.133:0/3017622638 wait complete. 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 Processor -- start 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 -- start start 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af4104aa0 0x7f3af419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af41a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:20.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410f580 con 0x7f3af4104aa0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410f400 con 0x7f3af41038b0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.898+0000 7f3af9000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410f700 con 0x7f3af4105460 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3aebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af4104aa0 0x7f3af419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:60354/0 (socket says 172.21.15.133:60354) 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af41a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 -- 172.21.15.133:0/1201154105 learned_addr learned my addr 172.21.15.133:0/1201154105 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 -- 172.21.15.133:0/1201154105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 msgr2=0x7f3af41a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af41a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 -- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af4104aa0 msgr2=0x7f3af419a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af4104aa0 0x7f3af419a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 -- 172.21.15.133:0/1201154105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af41a17b0 con 0x7f3af41038b0 2024-05-09T08:27:20.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.899+0000 7f3af3400640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400e9c0 tx=0x7f3ae400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.900+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400cd80 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.900+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae40035f0 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.900+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af41a1aa0 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.900+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4010690 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.900+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a2040 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.901+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af410ab70 con 0x7f3af41038b0 2024-05-09T08:27:20.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.903+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3ae40037a0 con 0x7f3af41038b0 2024-05-09T08:27:20.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.904+0000 7f3af0c00640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 0x7f3ad80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:20.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.904+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3ae40cefa0 con 0x7f3af41038b0 2024-05-09T08:27:20.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.904+0000 7f3aebe00640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 0x7f3ad80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:20.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.905+0000 7f3aebe00640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 0x7f3ad80a50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3af419b0a0 tx=0x7f3adc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:20.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:20.908+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ae409c300 con 0x7f3af41038b0 2024-05-09T08:27:20.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:20 smithi188 ceph-mon[34645]: Deploying daemon mds.d on smithi133 2024-05-09T08:27:20.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:20 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6834/1799489914,v1:172.21.15.133:6835/1799489914] up:boot 2024-05-09T08:27:20.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:20 smithi188 ceph-mon[34645]: fsmap 3 up:standby 2024-05-09T08:27:20.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:20 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T08:27:21.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.156+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3af402d1c0 con 0x7f3af41038b0 2024-05-09T08:27:21.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.157+0000 7f3af0c00640 1 -- 172.21.15.133:0/1201154105 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f3ae409bc00 con 0x7f3af41038b0 2024-05-09T08:27:21.159 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:21.159 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:27:21.159 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 4 2024-05-09T08:27:21.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 msgr2=0x7f3ad80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:21.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 0x7f3ad80a50d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3af419b0a0 tx=0x7f3adc005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 msgr2=0x7f3af4199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400e9c0 tx=0x7f3ae400ee80 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 shutdown_connections 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ad80a2c10 0x7f3ad80a50d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af4105460 0x7f3af41a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3af4104aa0 0x7f3af419a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 --2- 172.21.15.133:0/1201154105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3af41038b0 0x7f3af4199b90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.160+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 >> 172.21.15.133:0/1201154105 conn(0x7f3af40fee40 msgr2=0x7f3af4107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.161+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 shutdown_connections 2024-05-09T08:27:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:21.161+0000 7f3af9000640 1 -- 172.21.15.133:0/1201154105 wait complete. 2024-05-09T08:27:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:20 smithi160 ceph-mon[34070]: Deploying daemon mds.d on smithi133 2024-05-09T08:27:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:20 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6834/1799489914,v1:172.21.15.133:6835/1799489914] up:boot 2024-05-09T08:27:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:20 smithi160 ceph-mon[34070]: fsmap 3 up:standby 2024-05-09T08:27:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:20 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T08:27:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:21 smithi160 ceph-mon[34070]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:21 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1201154105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:22.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:21 smithi133 ceph-mon[29610]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:22.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:21 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1201154105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:21 smithi188 ceph-mon[34645]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:21 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1201154105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:22.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:22 smithi133 ceph-mon[29610]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:22 smithi160 ceph-mon[34070]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:23.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:22 smithi188 ceph-mon[34645]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:23.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 -- 172.21.15.133:0/3132475893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d440761f0 msgr2=0x7f6d44076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:23.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 --2- 172.21.15.133:0/3132475893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d440761f0 0x7f6d44076610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009ca0 tx=0x7f6d3402a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 -- 172.21.15.133:0/3132475893 shutdown_connections 2024-05-09T08:27:23.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 --2- 172.21.15.133:0/3132475893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 0x7f6d44078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 --2- 172.21.15.133:0/3132475893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d44077430 0x7f6d44077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 --2- 172.21.15.133:0/3132475893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d440761f0 0x7f6d44076610 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 -- 172.21.15.133:0/3132475893 >> 172.21.15.133:0/3132475893 conn(0x7f6d440ff020 msgr2=0x7f6d44101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 -- 172.21.15.133:0/3132475893 shutdown_connections 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.379+0000 7f6d49a00640 1 -- 172.21.15.133:0/3132475893 wait complete. 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 Processor -- start 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 -- start start 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 0x7f6d4419e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:23.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 0x7f6d441a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113b40 con 0x7f6d44077430 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441139c0 con 0x7f6d440761f0 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.380+0000 7f6d49a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44113cc0 con 0x7f6d44077dd0 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43524/0 (socket says 172.21.15.133:43524) 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 -- 172.21.15.133:0/2954976460 learned_addr learned my addr 172.21.15.133:0/2954976460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:23.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 0x7f6d4419e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d43400640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 0x7f6d441a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 -- 172.21.15.133:0/2954976460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 msgr2=0x7f6d441a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 0x7f6d441a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 -- 172.21.15.133:0/2954976460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 msgr2=0x7f6d4419e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 0x7f6d4419e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 -- 172.21.15.133:0/2954976460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441a5dd0 con 0x7f6d44077430 2024-05-09T08:27:23.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 0x7f6d4419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d42000640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9f0 tx=0x7f6d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.381+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00ccc0 con 0x7f6d44077430 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.382+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c010910 con 0x7f6d44077430 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.382+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0188a0 con 0x7f6d44077430 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.382+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441a60c0 con 0x7f6d44077430 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.382+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a65c0 con 0x7f6d44077430 2024-05-09T08:27:23.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.383+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44110600 con 0x7f6d44077430 2024-05-09T08:27:23.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.385+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6d2c010ac0 con 0x7f6d44077430 2024-05-09T08:27:23.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.386+0000 7f6d3b400640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 0x7f6d280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:23.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.386+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6d2c0cf0a0 con 0x7f6d44077430 2024-05-09T08:27:23.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.386+0000 7f6d42a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 0x7f6d280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:23.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.387+0000 7f6d42a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 0x7f6d280a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d34031c10 tx=0x7f6d34009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:23.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.391+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6d2c09c400 con 0x7f6d44077430 2024-05-09T08:27:23.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.642+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4402d1c0 con 0x7f6d44077430 2024-05-09T08:27:23.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.644+0000 7f6d3b400640 1 -- 172.21.15.133:0/2954976460 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f6d2c09bd00 con 0x7f6d44077430 2024-05-09T08:27:23.645 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:23.645 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:27:23.645 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 4 2024-05-09T08:27:23.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 msgr2=0x7f6d280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 0x7f6d280a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d34031c10 tx=0x7f6d34009360 comp rx=0 tx=0).stop 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 msgr2=0x7f6d4419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9f0 tx=0x7f6d2c00edf0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 shutdown_connections 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6d280a2c10 0x7f6d280a50d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44077dd0 0x7f6d441a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d44077430 0x7f6d4419e6f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 --2- 172.21.15.133:0/2954976460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d440761f0 0x7f6d4419e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 >> 172.21.15.133:0/2954976460 conn(0x7f6d440ff020 msgr2=0x7f6d44100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 shutdown_connections 2024-05-09T08:27:23.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:23.647+0000 7f6d49a00640 1 -- 172.21.15.133:0/2954976460 wait complete. 2024-05-09T08:27:23.665 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:27:23 smithi133 systemd[1]: Started Ceph mds.d for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: Deploying daemon mds.e on smithi160 2024-05-09T08:27:24.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2954976460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:24.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6836/3539634696,v1:172.21.15.133:6837/3539634696] up:boot 2024-05-09T08:27:24.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: fsmap 4 up:standby 2024-05-09T08:27:24.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:24 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: Deploying daemon mds.e on smithi160 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2954976460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6836/3539634696,v1:172.21.15.133:6837/3539634696] up:boot 2024-05-09T08:27:24.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: fsmap 4 up:standby 2024-05-09T08:27:24.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:24 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T08:27:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: Deploying daemon mds.e on smithi160 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2954976460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6836/3539634696,v1:172.21.15.133:6837/3539634696] up:boot 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: fsmap 4 up:standby 2024-05-09T08:27:24.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:24 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T08:27:25.668 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:25.857 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:25 smithi160 ceph-mon[34070]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 -- 172.21.15.133:0/3825690675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 msgr2=0x7f0e24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3825690675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e24106e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009900 tx=0x7f0e1002a410 comp rx=0 tx=0).stop 2024-05-09T08:27:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 -- 172.21.15.133:0/3825690675 shutdown_connections 2024-05-09T08:27:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3825690675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24107390 0x7f0e241097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3825690675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e24106e50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3825690675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24102130 0x7f0e24104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 -- 172.21.15.133:0/3825690675 >> 172.21.15.133:0/3825690675 conn(0x7f0e240fbb00 msgr2=0x7f0e240fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 -- 172.21.15.133:0/3825690675 shutdown_connections 2024-05-09T08:27:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.879+0000 7f0e2b000640 1 -- 172.21.15.133:0/3825690675 wait complete. 2024-05-09T08:27:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.880+0000 7f0e2b000640 1 Processor -- start 2024-05-09T08:27:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 -- start start 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e2419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24107390 0x7f0e241a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410d370 con 0x7f0e24107390 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410d1f0 con 0x7f0e24104a60 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410d4f0 con 0x7f0e24102130 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e29200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e2419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e29c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e29200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e2419a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:35394/0 (socket says 172.21.15.133:35394) 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e29c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:56600/0 (socket says 172.21.15.133:56600) 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e29200640 1 -- 172.21.15.133:0/3892683252 learned_addr learned my addr 172.21.15.133:0/3892683252 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.881+0000 7f0e2a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24107390 0x7f0e241a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 -- 172.21.15.133:0/3892683252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 msgr2=0x7f0e2419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e2419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 -- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24107390 msgr2=0x7f0e241a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24107390 0x7f0e241a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 -- 172.21.15.133:0/3892683252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241a1850 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e29c00640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e9f0 tx=0x7f0e1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800ccc0 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241a1b40 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e18010910 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.882+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18018790 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.883+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a20e0 con 0x7f0e24102130 2024-05-09T08:27:25.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.884+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e24108ee0 con 0x7f0e24102130 2024-05-09T08:27:25.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.885+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0e18010ac0 con 0x7f0e24102130 2024-05-09T08:27:25.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.887+0000 7f0e22000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.887+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0e180cf060 con 0x7f0e24102130 2024-05-09T08:27:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.887+0000 7f0e29200640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:25.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.888+0000 7f0e29200640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000c00 tx=0x7f0e100315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:25.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:25.891+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0e1809c3c0 con 0x7f0e24102130 2024-05-09T08:27:25.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:25 smithi133 ceph-mon[29610]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:25.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:25 smithi188 ceph-mon[34645]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:26.110 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 podman[63821]: 2024-05-09 08:27:25.977498705 +0000 UTC m=+0.017833739 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:26.110 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 podman[63821]: 2024-05-09 08:27:26.110673545 +0000 UTC m=+0.151008572 container create 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T08:27:26.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.132+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e2402d1c0 con 0x7f0e24102130 2024-05-09T08:27:26.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.135+0000 7f0e22000640 1 -- 172.21.15.133:0/3892683252 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7f0e2402d1c0 con 0x7f0e24102130 2024-05-09T08:27:26.135 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:26.135 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:27:26.135 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 5 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.136+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 msgr2=0x7f0e0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.136+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000c00 tx=0x7f0e100315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.136+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 msgr2=0x7f0e24199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.136+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800e9f0 tx=0x7f0e1800edf0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 shutdown_connections 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0e0c0a2b90 0x7f0e0c0a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e24107390 0x7f0e241a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e24104a60 0x7f0e2419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 --2- 172.21.15.133:0/3892683252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e24102130 0x7f0e24199c40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 >> 172.21.15.133:0/3892683252 conn(0x7f0e240fbb00 msgr2=0x7f0e24101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:26.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 shutdown_connections 2024-05-09T08:27:26.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:26.138+0000 7f0e2b000640 1 -- 172.21.15.133:0/3892683252 wait complete. 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:26 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3892683252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 podman[63821]: 2024-05-09 08:27:26.493299903 +0000 UTC m=+0.533634928 container init 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD) 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 podman[63821]: 2024-05-09 08:27:26.495959444 +0000 UTC m=+0.536294471 container start 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 ceph-mds[63841]: -- 172.21.15.160:0/3174132567 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5dbd95a40 con 0x55c5dbd68c00 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e[63837]: starting mds.e at 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 bash[63821]: 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 2024-05-09T08:27:26.913 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:27:26 smithi160 systemd[1]: Started Ceph mds.e for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:26.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:26 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3892683252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:26.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:26 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3892683252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:27.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: Deploying daemon mds.f on smithi188 2024-05-09T08:27:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6834/4102808369,v1:172.21.15.160:6835/4102808369] up:boot 2024-05-09T08:27:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: fsmap 5 up:standby 2024-05-09T08:27:27.914 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:27.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: Deploying daemon mds.f on smithi188 2024-05-09T08:27:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6834/4102808369,v1:172.21.15.160:6835/4102808369] up:boot 2024-05-09T08:27:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: fsmap 5 up:standby 2024-05-09T08:27:27.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: Deploying daemon mds.f on smithi188 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6834/4102808369,v1:172.21.15.160:6835/4102808369] up:boot 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: fsmap 5 up:standby 2024-05-09T08:27:27.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T08:27:28.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 -- 172.21.15.133:0/1982817496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1808074290 msgr2=0x7f1808074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 --2- 172.21.15.133:0/1982817496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1808074290 0x7f1808074710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009890 tx=0x7f17f402a360 comp rx=0 tx=0).stop 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 -- 172.21.15.133:0/1982817496 shutdown_connections 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 --2- 172.21.15.133:0/1982817496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f180810de70 0x7f1808110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 --2- 172.21.15.133:0/1982817496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1808074290 0x7f1808074710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 --2- 172.21.15.133:0/1982817496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808075870 0x7f1808073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 -- 172.21.15.133:0/1982817496 >> 172.21.15.133:0/1982817496 conn(0x7f18080fce10 msgr2=0x7f18080ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 -- 172.21.15.133:0/1982817496 shutdown_connections 2024-05-09T08:27:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.373+0000 7f180fa00640 1 -- 172.21.15.133:0/1982817496 wait complete. 2024-05-09T08:27:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.374+0000 7f180fa00640 1 Processor -- start 2024-05-09T08:27:28.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.374+0000 7f180fa00640 1 -- start start 2024-05-09T08:27:28.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 0x7f180819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:28.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:28.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 0x7f18081a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:28.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1808113af0 con 0x7f1808074290 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1808113970 con 0x7f180810de70 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1808113c70 con 0x7f1808075870 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 0x7f180819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f1807e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 0x7f180819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46020/0 (socket says 172.21.15.133:46020) 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180d000640 1 -- 172.21.15.133:0/875783275 learned_addr learned my addr 172.21.15.133:0/875783275 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f1807e00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:56622/0 (socket says 172.21.15.133:56622) 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.375+0000 7f180da00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 0x7f18081a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 -- 172.21.15.133:0/875783275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 msgr2=0x7f18081a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 0x7f18081a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 -- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 msgr2=0x7f180819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:28.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 0x7f180819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 -- 172.21.15.133:0/875783275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081a5df0 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f180da00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 0x7f18081a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1807e00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17f402a880 tx=0x7f17f4033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f4037040 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081a6080 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f4033e10 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.376+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f40046c0 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.377+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081a65c0 con 0x7f1808075870 2024-05-09T08:27:28.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.377+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1808104040 con 0x7f1808075870 2024-05-09T08:27:28.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.380+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f17f4004050 con 0x7f1808075870 2024-05-09T08:27:28.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.381+0000 7f1805600640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 0x7f17f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:28.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.381+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f17f40eeab0 con 0x7f1808075870 2024-05-09T08:27:28.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.381+0000 7f180d000640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 0x7f17f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:28.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.382+0000 7f180d000640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 0x7f17f00a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009fd0 tx=0x7f17fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:28.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.383+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f17f40bbe90 con 0x7f1808075870 2024-05-09T08:27:28.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.623+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f180802d1c0 con 0x7f1808075870 2024-05-09T08:27:28.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.624+0000 7f1805600640 1 -- 172.21.15.133:0/875783275 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4708 (secure 0 0 0) 0x7f17f40bb790 con 0x7f1808075870 2024-05-09T08:27:28.624 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:28.624 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T08:27:28.624 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 6 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 msgr2=0x7f17f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 0x7f17f00a5050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009fd0 tx=0x7f17fc009500 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 msgr2=0x7f180819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17f402a880 tx=0x7f17f4033b60 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 shutdown_connections 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f17f00a2b90 0x7f17f00a5050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f180810de70 0x7f18081a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1808075870 0x7f180819e710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 --2- 172.21.15.133:0/875783275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1808074290 0x7f180819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.626+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 >> 172.21.15.133:0/875783275 conn(0x7f18080fce10 msgr2=0x7f1808101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.627+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 shutdown_connections 2024-05-09T08:27:28.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:28.627+0000 7f180fa00640 1 -- 172.21.15.133:0/875783275 wait complete. 2024-05-09T08:27:28.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:28 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/875783275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:28.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:28 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/875783275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:29.101 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:28 smithi188 systemd[1]: Starting Ceph mds.f for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:29.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:28 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/875783275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:29.371 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 podman[65177]: 2024-05-09 08:27:29.003912614 +0000 UTC m=+0.017929603 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:29.372 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 podman[65177]: 2024-05-09 08:27:29.10977635 +0000 UTC m=+0.123793326 container create f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T08:27:29.372 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 podman[65177]: 2024-05-09 08:27:29.346205165 +0000 UTC m=+0.360222143 container init f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-09T08:27:29.372 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 podman[65177]: 2024-05-09 08:27:29.348714776 +0000 UTC m=+0.362731755 container start f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-96805dc, GIT_CLEAN=True) 2024-05-09T08:27:29.372 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mds[65196]: -- 172.21.15.188:0/3127094319 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560bbb6a5a40 con 0x560bbb678c00 2024-05-09T08:27:29.667 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f[65192]: starting mds.f at 2024-05-09T08:27:29.667 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 bash[65177]: f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b 2024-05-09T08:27:29.668 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:27:29 smithi188 systemd[1]: Started Ceph mds.f for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:29.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:29.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:29.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:29.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:30.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:30.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 -- 172.21.15.133:0/1950517527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0108ea0 msgr2=0x7ff8f0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 --2- 172.21.15.133:0/1950517527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0108ea0 0x7ff8f0109320 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009900 tx=0x7ff8d802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 -- 172.21.15.133:0/1950517527 shutdown_connections 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 --2- 172.21.15.133:0/1950517527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 0x7ff8f01100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 --2- 172.21.15.133:0/1950517527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0108ea0 0x7ff8f0109320 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.861+0000 7ff8f5000640 1 --2- 172.21.15.133:0/1950517527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0107cb0 0x7ff8f01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.862+0000 7ff8f5000640 1 -- 172.21.15.133:0/1950517527 >> 172.21.15.133:0/1950517527 conn(0x7ff8f0076c70 msgr2=0x7ff8f0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.862+0000 7ff8f5000640 1 -- 172.21.15.133:0/1950517527 shutdown_connections 2024-05-09T08:27:30.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.862+0000 7ff8f5000640 1 -- 172.21.15.133:0/1950517527 wait complete. 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 Processor -- start 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 -- start start 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0108ea0 0x7ff8f019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 0x7ff8f01a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0113960 con 0x7ff8f0109860 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01137e0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.863+0000 7ff8f5000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f0113ae0 con 0x7ff8f0108ea0 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:35448/0 (socket says 172.21.15.133:35448) 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 -- 172.21.15.133:0/4085077603 learned_addr learned my addr 172.21.15.133:0/4085077603 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:30.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8efe00640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 0x7ff8f01a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8e6a00640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0108ea0 0x7ff8f019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 -- 172.21.15.133:0/4085077603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0108ea0 msgr2=0x7ff8f019e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0108ea0 0x7ff8f019e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 -- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 msgr2=0x7ff8f01a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 0x7ff8f01a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:30.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 -- 172.21.15.133:0/4085077603 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f01a5bc0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.864+0000 7ff8ef400640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9c0 tx=0x7ff8e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.865+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cde0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.865+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e0010910 con 0x7ff8f0107cb0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.865+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0018a00 con 0x7ff8f0107cb0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.865+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01a5eb0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.865+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f01a62c0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.866+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f010efd0 con 0x7ff8f0107cb0 2024-05-09T08:27:30.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.868+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff8e001f020 con 0x7ff8f0107cb0 2024-05-09T08:27:30.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.869+0000 7ff8ecc00640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:30.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.869+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8e00ced80 con 0x7ff8f0107cb0 2024-05-09T08:27:30.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.869+0000 7ff8e6a00640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:30.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.870+0000 7ff8e6a00640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 0x7ff8d40a5150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8031440 tx=0x7ff8d80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:30.874+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8e009c0e0 con 0x7ff8f0107cb0 2024-05-09T08:27:31.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.112+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f00624d0 con 0x7ff8f0107cb0 2024-05-09T08:27:31.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.113+0000 7ff8ecc00640 1 -- 172.21.15.133:0/4085077603 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7ff8e009b9e0 con 0x7ff8f0107cb0 2024-05-09T08:27:31.113 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:31.114 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-05-09T08:27:31.114 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 msgr2=0x7ff8d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 0x7ff8d40a5150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8031440 tx=0x7ff8d80315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 msgr2=0x7ff8f019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9c0 tx=0x7ff8e000ee80 comp rx=0 tx=0).stop 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 shutdown_connections 2024-05-09T08:27:31.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff8d40a2c90 0x7ff8d40a5150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8f0109860 0x7ff8f01a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8f0108ea0 0x7ff8f019e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 --2- 172.21.15.133:0/4085077603 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8f0107cb0 0x7ff8f019dfa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 >> 172.21.15.133:0/4085077603 conn(0x7ff8f0076c70 msgr2=0x7ff8f010c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 shutdown_connections 2024-05-09T08:27:31.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:31.116+0000 7ff8f5000640 1 -- 172.21.15.133:0/4085077603 wait complete. 2024-05-09T08:27:31.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:30 smithi133 ceph-mon[29610]: Deploying daemon mds.c on smithi188 2024-05-09T08:27:31.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:30 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6834/232300149,v1:172.21.15.188:6835/232300149] up:boot 2024-05-09T08:27:31.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:30 smithi133 ceph-mon[29610]: fsmap 6 up:standby 2024-05-09T08:27:31.132 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T08:27:31.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:30 smithi160 ceph-mon[34070]: Deploying daemon mds.c on smithi188 2024-05-09T08:27:31.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:30 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6834/232300149,v1:172.21.15.188:6835/232300149] up:boot 2024-05-09T08:27:31.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:30 smithi160 ceph-mon[34070]: fsmap 6 up:standby 2024-05-09T08:27:31.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T08:27:31.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:30 smithi188 ceph-mon[34645]: Deploying daemon mds.c on smithi188 2024-05-09T08:27:31.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:30 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6834/232300149,v1:172.21.15.188:6835/232300149] up:boot 2024-05-09T08:27:31.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:30 smithi188 ceph-mon[34645]: fsmap 6 up:standby 2024-05-09T08:27:31.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T08:27:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:31 smithi188 ceph-mon[34645]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:31.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:31 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4085077603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:31.933 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:31 smithi188 systemd[1]: Starting Ceph mds.c for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:32.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:31 smithi160 ceph-mon[34070]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:32.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:31 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4085077603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:32.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:31 smithi133 ceph-mon[29610]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:32.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:31 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4085077603' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:32.234 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:31 smithi188 podman[65484]: 2024-05-09 08:27:31.856226141 +0000 UTC m=+0.018246409 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:32.234 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:31 smithi188 podman[65484]: 2024-05-09 08:27:31.990440494 +0000 UTC m=+0.152460762 container create d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T08:27:32.234 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 podman[65484]: 2024-05-09 08:27:32.235077381 +0000 UTC m=+0.397097649 container init d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:27:32.683 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 podman[65484]: 2024-05-09 08:27:32.237813563 +0000 UTC m=+0.399833834 container start d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T08:27:32.683 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 ceph-mds[65504]: -- 172.21.15.188:0/677983954 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56248b875a40 con 0x56248b848c00 2024-05-09T08:27:32.683 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: starting mds.c at 2024-05-09T08:27:32.683 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 bash[65484]: d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 2024-05-09T08:27:32.683 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:27:32 smithi188 systemd[1]: Started Ceph mds.c for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:33.135 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 -- 172.21.15.133:0/465149437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760108ea0 msgr2=0x7fe760109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 --2- 172.21.15.133:0/465149437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760108ea0 0x7fe760109320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a410 comp rx=0 tx=0).stop 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 -- 172.21.15.133:0/465149437 shutdown_connections 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 --2- 172.21.15.133:0/465149437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760109860 0x7fe7601100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 --2- 172.21.15.133:0/465149437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760108ea0 0x7fe760109320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 --2- 172.21.15.133:0/465149437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760107cb0 0x7fe7601080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.344+0000 7fe767200640 1 -- 172.21.15.133:0/465149437 >> 172.21.15.133:0/465149437 conn(0x7fe760076c70 msgr2=0x7fe760079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.345+0000 7fe767200640 1 -- 172.21.15.133:0/465149437 shutdown_connections 2024-05-09T08:27:33.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.345+0000 7fe767200640 1 -- 172.21.15.133:0/465149437 wait complete. 2024-05-09T08:27:33.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 Processor -- start 2024-05-09T08:27:33.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 -- start start 2024-05-09T08:27:33.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:33.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:33.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760109860 0x7fe7601a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe760113960 con 0x7fe760107cb0 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601137e0 con 0x7fe760108ea0 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.346+0000 7fe767200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe760113ae0 con 0x7fe760109860 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46038/0 (socket says 172.21.15.133:46038) 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:35468/0 (socket says 172.21.15.133:35468) 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765e00640 1 -- 172.21.15.133:0/3033912100 learned_addr learned my addr 172.21.15.133:0/3033912100 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 -- 172.21.15.133:0/3033912100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760109860 msgr2=0x7fe7601a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760109860 0x7fe7601a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 -- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 msgr2=0x7fe76019dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765400640 1 -- 172.21.15.133:0/3033912100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7601a5bf0 con 0x7fe760108ea0 2024-05-09T08:27:33.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.347+0000 7fe765e00640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:33.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe765400640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe75002a930 tx=0x7fe750032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:33.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75003f030 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe750003ce0 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750035070 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7601a5e80 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.348+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7601a6290 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.349+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76010efd0 con 0x7fe760108ea0 2024-05-09T08:27:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.351+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe750042030 con 0x7fe760108ea0 2024-05-09T08:27:33.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.353+0000 7fe75aa00640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 0x7fe7480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:33.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.353+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe7500ef430 con 0x7fe760108ea0 2024-05-09T08:27:33.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.353+0000 7fe765e00640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 0x7fe7480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:33.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.353+0000 7fe765e00640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 0x7fe7480a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7540058c0 tx=0x7fe754005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:33.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.356+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7500bc790 con 0x7fe760108ea0 2024-05-09T08:27:33.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.596+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe760062530 con 0x7fe760108ea0 2024-05-09T08:27:33.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.597+0000 7fe75aa00640 1 -- 172.21.15.133:0/3033912100 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7fe7500bc090 con 0x7fe760108ea0 2024-05-09T08:27:33.597 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:33.598 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-09T08:27:33.598 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 msgr2=0x7fe7480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 0x7fe7480a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7540058c0 tx=0x7fe754005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 msgr2=0x7fe76019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe75002a930 tx=0x7fe750032ce0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 shutdown_connections 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.600+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe7480a2c10 0x7fe7480a50d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760109860 0x7fe7601a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe760108ea0 0x7fe76019e510 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 --2- 172.21.15.133:0/3033912100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe760107cb0 0x7fe76019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 >> 172.21.15.133:0/3033912100 conn(0x7fe760076c70 msgr2=0x7fe76010c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 shutdown_connections 2024-05-09T08:27:33.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:33.601+0000 7fe767200640 1 -- 172.21.15.133:0/3033912100 wait complete. 2024-05-09T08:27:33.616 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.616 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: Deploying daemon mds.h on smithi160 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6836/3112088995,v1:172.21.15.188:6837/3112088995] up:boot 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: fsmap 7 up:standby 2024-05-09T08:27:33.617 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: Deploying daemon mds.h on smithi160 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:33.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6836/3112088995,v1:172.21.15.188:6837/3112088995] up:boot 2024-05-09T08:27:33.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: fsmap 7 up:standby 2024-05-09T08:27:33.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: Deploying daemon mds.h on smithi160 2024-05-09T08:27:33.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:33.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6836/3112088995,v1:172.21.15.188:6837/3112088995] up:boot 2024-05-09T08:27:33.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: fsmap 7 up:standby 2024-05-09T08:27:33.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:27:34.831 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:34 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3033912100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:34.831 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:34 smithi160 systemd[1]: Starting Ceph mds.h for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:34.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:34 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3033912100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:34 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3033912100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:35.089 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:34 smithi160 podman[64133]: 2024-05-09 08:27:34.734400221 +0000 UTC m=+0.018195381 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:35.089 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:34 smithi160 podman[64133]: 2024-05-09 08:27:34.862410627 +0000 UTC m=+0.146205770 container create 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T08:27:35.089 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 podman[64133]: 2024-05-09 08:27:35.089550736 +0000 UTC m=+0.373345880 container init 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc) 2024-05-09T08:27:35.413 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 podman[64133]: 2024-05-09 08:27:35.09236141 +0000 UTC m=+0.376156558 container start 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=) 2024-05-09T08:27:35.413 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mds[64153]: -- 172.21.15.160:0/346618835 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562707983a40 con 0x562707956800 2024-05-09T08:27:35.413 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: starting mds.h at 2024-05-09T08:27:35.413 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 bash[64133]: 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b 2024-05-09T08:27:35.413 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:27:35 smithi160 systemd[1]: Started Ceph mds.h for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:35.620 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.828+0000 7f3b2fe00640 1 -- 172.21.15.133:0/1306889735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 msgr2=0x7f3b28103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.828+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/1306889735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28103ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009900 tx=0x7f3b1c02a410 comp rx=0 tx=0).stop 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 -- 172.21.15.133:0/1306889735 shutdown_connections 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/1306889735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b2810bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/1306889735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 0x7f3b28105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/1306889735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28103ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 -- 172.21.15.133:0/1306889735 >> 172.21.15.133:0/1306889735 conn(0x7f3b280ff020 msgr2=0x7f3b28101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 -- 172.21.15.133:0/1306889735 shutdown_connections 2024-05-09T08:27:35.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.829+0000 7f3b2fe00640 1 -- 172.21.15.133:0/1306889735 wait complete. 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 Processor -- start 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 -- start start 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 0x7f3b2819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b281a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2810f750 con 0x7f3b28105660 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2810f5d0 con 0x7f3b28104ca0 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2810f8d0 con 0x7f3b28103ab0 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b281a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:35.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:57530/0 (socket says 172.21.15.133:57530) 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2d400640 1 -- 172.21.15.133:0/3290914046 learned_addr learned my addr 172.21.15.133:0/3290914046 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.830+0000 7f3b2ca00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 0x7f3b2819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 -- 172.21.15.133:0/3290914046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 msgr2=0x7f3b2819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 0x7f3b2819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 -- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 msgr2=0x7f3b281a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b281a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 -- 172.21.15.133:0/3290914046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b281a1a30 con 0x7f3b28103ab0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2de00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b281a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2d400640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c031440 tx=0x7f3b1c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c03f030 con 0x7f3b28103ab0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b281a1cc0 con 0x7f3b28103ab0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.831+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c003ce0 con 0x7f3b28103ab0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.832+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c035070 con 0x7f3b28103ab0 2024-05-09T08:27:35.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.832+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b281a21b0 con 0x7f3b28103ab0 2024-05-09T08:27:35.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.833+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2810add0 con 0x7f3b28103ab0 2024-05-09T08:27:35.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.834+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b1c042030 con 0x7f3b28103ab0 2024-05-09T08:27:35.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.836+0000 7f3b26000640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:35.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.836+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3b1c0ef110 con 0x7f3b28103ab0 2024-05-09T08:27:35.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.836+0000 7f3b2ca00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:35.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.837+0000 7f3b2ca00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 0x7f3b100a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3b2819b320 tx=0x7f3b14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:35.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:35.840+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3b1c0bc470 con 0x7f3b28103ab0 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:35.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:35.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:35.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:35.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:35.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.078+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b28062430 con 0x7f3b28103ab0 2024-05-09T08:27:36.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.079+0000 7f3b26000640 1 -- 172.21.15.133:0/3290914046 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f3b1c0bbd70 con 0x7f3b28103ab0 2024-05-09T08:27:36.080 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:36.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T08:27:36.080 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 9 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 msgr2=0x7f3b100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 0x7f3b100a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3b2819b320 tx=0x7f3b14005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 msgr2=0x7f3b28199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c031440 tx=0x7f3b1c032ce0 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 shutdown_connections 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3b100a2b90 0x7f3b100a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b28105660 0x7f3b281a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b28104ca0 0x7f3b2819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 --2- 172.21.15.133:0/3290914046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b28103ab0 0x7f3b28199e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 >> 172.21.15.133:0/3290914046 conn(0x7f3b280ff020 msgr2=0x7f3b28107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:36.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 shutdown_connections 2024-05-09T08:27:36.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:36.083+0000 7f3b2fe00640 1 -- 172.21.15.133:0/3290914046 wait complete. 2024-05-09T08:27:36.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:36 smithi160 ceph-mon[34070]: Deploying daemon mds.i on smithi188 2024-05-09T08:27:36.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:36 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6836/1342622403,v1:172.21.15.160:6837/1342622403] up:boot 2024-05-09T08:27:36.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:36 smithi160 ceph-mon[34070]: fsmap 8 up:standby 2024-05-09T08:27:36.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:27:36.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:36 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3290914046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:36 smithi133 ceph-mon[29610]: Deploying daemon mds.i on smithi188 2024-05-09T08:27:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:36 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6836/1342622403,v1:172.21.15.160:6837/1342622403] up:boot 2024-05-09T08:27:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:36 smithi133 ceph-mon[29610]: fsmap 8 up:standby 2024-05-09T08:27:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:27:36.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:36 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3290914046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:36.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:36 smithi188 ceph-mon[34645]: Deploying daemon mds.i on smithi188 2024-05-09T08:27:36.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:36 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6836/1342622403,v1:172.21.15.160:6837/1342622403] up:boot 2024-05-09T08:27:36.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:36 smithi188 ceph-mon[34645]: fsmap 8 up:standby 2024-05-09T08:27:36.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:27:36.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:36 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3290914046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:37.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:37 smithi188 ceph-mon[34645]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:37.683 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:37 smithi188 systemd[1]: Starting Ceph mds.i for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:37.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:37 smithi160 ceph-mon[34070]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:37.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:37 smithi133 ceph-mon[29610]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:38.031 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:37 smithi188 podman[65791]: 2024-05-09 08:27:37.679079762 +0000 UTC m=+0.018031535 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:38.032 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:37 smithi188 podman[65791]: 2024-05-09 08:27:37.796190093 +0000 UTC m=+0.135141852 container create 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T08:27:38.032 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 podman[65791]: 2024-05-09 08:27:38.032300516 +0000 UTC m=+0.371252274 container init 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T08:27:38.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 -- 172.21.15.133:0/3743795783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc075870 msgr2=0x7f07cc073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 --2- 172.21.15.133:0/3743795783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc075870 0x7f07cc073cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009900 tx=0x7f07bc02a410 comp rx=0 tx=0).stop 2024-05-09T08:27:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 -- 172.21.15.133:0/3743795783 shutdown_connections 2024-05-09T08:27:38.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 --2- 172.21.15.133:0/3743795783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc109740 0x7f07cc10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 --2- 172.21.15.133:0/3743795783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc074290 0x7f07cc074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 --2- 172.21.15.133:0/3743795783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc075870 0x7f07cc073cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 -- 172.21.15.133:0/3743795783 >> 172.21.15.133:0/3743795783 conn(0x7f07cc0fce10 msgr2=0x7f07cc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 -- 172.21.15.133:0/3743795783 shutdown_connections 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.313+0000 7f07d2600640 1 -- 172.21.15.133:0/3743795783 wait complete. 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 Processor -- start 2024-05-09T08:27:38.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 -- start start 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc074290 0x7f07cc199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 0x7f07cc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f710 con 0x7f07cc109740 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f590 con 0x7f07cc074290 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07d2600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc10f890 con 0x7f07cc075870 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07cbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07cbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59580/0 (socket says 172.21.15.133:59580) 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07caa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 0x7f07cc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:38.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.315+0000 7f07cbe00640 1 -- 172.21.15.133:0/1768883173 learned_addr learned my addr 172.21.15.133:0/1768883173 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cb400640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc074290 0x7f07cc199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 -- 172.21.15.133:0/1768883173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 msgr2=0x7f07cc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 0x7f07cc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 -- 172.21.15.133:0/1768883173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc074290 msgr2=0x7f07cc199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc074290 0x7f07cc199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 -- 172.21.15.133:0/1768883173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc1a1990 con 0x7f07cc109740 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07caa00640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 0x7f07cc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:38.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.316+0000 7f07cbe00640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07c400eb10 tx=0x7f07c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.317+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4004b20 con 0x7f07cc109740 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.317+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4004cc0 con 0x7f07cc109740 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.317+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1a1c80 con 0x7f07cc109740 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.317+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4010760 con 0x7f07cc109740 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.317+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc1a21d0 con 0x7f07cc109740 2024-05-09T08:27:38.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.318+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc108470 con 0x7f07cc109740 2024-05-09T08:27:38.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.320+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f07c4010900 con 0x7f07cc109740 2024-05-09T08:27:38.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.322+0000 7f07c3e00640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 0x7f07b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.322+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f07c40cf350 con 0x7f07cc109740 2024-05-09T08:27:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.322+0000 7f07cb400640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 0x7f07b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:38.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.323+0000 7f07cb400640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 0x7f07b00a50d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07bc031440 tx=0x7f07bc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:38.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.325+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07c4053740 con 0x7f07cc109740 2024-05-09T08:27:38.433 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 podman[65791]: 2024-05-09 08:27:38.035184827 +0000 UTC m=+0.374136593 container start 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T08:27:38.433 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 ceph-mds[65810]: -- 172.21.15.188:0/1414387921 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575f8015a40 con 0x5575f7fe8800 2024-05-09T08:27:38.433 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: starting mds.i at 2024-05-09T08:27:38.433 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 bash[65791]: 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 2024-05-09T08:27:38.433 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:27:38 smithi188 systemd[1]: Started Ceph mds.i for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.567+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07cc02d410 con 0x7f07cc109740 2024-05-09T08:27:38.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.568+0000 7f07c3e00640 1 -- 172.21.15.133:0/1768883173 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f07c409c6b0 con 0x7f07cc109740 2024-05-09T08:27:38.568 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:38.568 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T08:27:38.568 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-05-09T08:27:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.570+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 msgr2=0x7f07b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:38.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.570+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 0x7f07b00a50d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07bc031440 tx=0x7f07bc0315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.570+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 msgr2=0x7f07cc1a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.570+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07c400eb10 tx=0x7f07c400c2d0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 shutdown_connections 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f07b00a2c10 0x7f07b00a50d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07cc109740 0x7f07cc1a1320 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07cc075870 0x7f07cc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 --2- 172.21.15.133:0/1768883173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07cc074290 0x7f07cc199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:38.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 >> 172.21.15.133:0/1768883173 conn(0x7f07cc0fce10 msgr2=0x7f07cc105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:38.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 shutdown_connections 2024-05-09T08:27:38.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:38.571+0000 7f07d2600640 1 -- 172.21.15.133:0/1768883173 wait complete. 2024-05-09T08:27:39.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: Deploying daemon mds.g on smithi133 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6838/2528648460,v1:172.21.15.188:6839/2528648460] up:boot 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: fsmap 9 up:standby 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1768883173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:39.429 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:39 smithi133 ceph-mon[29610]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:39.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.454 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: Deploying daemon mds.g on smithi133 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6838/2528648460,v1:172.21.15.188:6839/2528648460] up:boot 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: fsmap 9 up:standby 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1768883173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:39.455 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:39 smithi188 ceph-mon[34645]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:39.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: Deploying daemon mds.g on smithi133 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6838/2528648460,v1:172.21.15.188:6839/2528648460] up:boot 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: fsmap 9 up:standby 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1768883173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:39.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:39 smithi160 ceph-mon[34070]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:40.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 -- 172.21.15.133:0/733733931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07b840 msgr2=0x7f9a4c07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 --2- 172.21.15.133:0/733733931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07b840 0x7f9a4c07bca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009900 tx=0x7f9a3802a410 comp rx=0 tx=0).stop 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 -- 172.21.15.133:0/733733931 shutdown_connections 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 --2- 172.21.15.133:0/733733931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c080aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 --2- 172.21.15.133:0/733733931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07b840 0x7f9a4c07bca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 --2- 172.21.15.133:0/733733931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07a600 0x7f9a4c07aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.086+0000 7f9a50a00640 1 -- 172.21.15.133:0/733733931 >> 172.21.15.133:0/733733931 conn(0x7f9a4c076e40 msgr2=0x7f9a4c079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.087+0000 7f9a50a00640 1 -- 172.21.15.133:0/733733931 shutdown_connections 2024-05-09T08:27:41.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.087+0000 7f9a50a00640 1 -- 172.21.15.133:0/733733931 wait complete. 2024-05-09T08:27:41.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.088+0000 7f9a50a00640 1 Processor -- start 2024-05-09T08:27:41.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.088+0000 7f9a50a00640 1 -- start start 2024-05-09T08:27:41.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 0x7f9a4c070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 0x7f9a4c071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c06b5e0 con 0x7f9a4c07b840 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c06b460 con 0x7f9a4c07a600 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a50a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c06b760 con 0x7f9a4c07c1e0 2024-05-09T08:27:41.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a4a800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a49e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 0x7f9a4c070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a4a800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:57574/0 (socket says 172.21.15.133:57574) 2024-05-09T08:27:41.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.089+0000 7f9a4a800640 1 -- 172.21.15.133:0/1728834614 learned_addr learned my addr 172.21.15.133:0/1728834614 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:41.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.090+0000 7f9a49400640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 0x7f9a4c071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:41.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49400640 1 -- 172.21.15.133:0/1728834614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 msgr2=0x7f9a4c0724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49400640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49400640 1 -- 172.21.15.133:0/1728834614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 msgr2=0x7f9a4c070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49400640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 0x7f9a4c070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49400640 1 -- 172.21.15.133:0/1728834614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c072b10 con 0x7f9a4c07b840 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.091+0000 7f9a49e00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 0x7f9a4c070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.092+0000 7f9a4a800640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.092+0000 7f9a49400640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 0x7f9a4c071460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a3802a930 tx=0x7f9a38032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:41.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.093+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3803f030 con 0x7f9a4c07b840 2024-05-09T08:27:41.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.093+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a38003ce0 con 0x7f9a4c07b840 2024-05-09T08:27:41.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.093+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38035070 con 0x7f9a4c07b840 2024-05-09T08:27:41.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.094+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c072da0 con 0x7f9a4c07b840 2024-05-09T08:27:41.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.095+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c073210 con 0x7f9a4c07b840 2024-05-09T08:27:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.098+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c07f8b0 con 0x7f9a4c07b840 2024-05-09T08:27:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.100+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9a38042030 con 0x7f9a4c07b840 2024-05-09T08:27:41.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.101+0000 7f9a42a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 0x7f9a300a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:41.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.102+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9a380eee80 con 0x7f9a4c07b840 2024-05-09T08:27:41.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.102+0000 7f9a49e00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 0x7f9a300a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.103+0000 7f9a49e00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 0x7f9a300a51a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005d00 tx=0x7f9a3c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.110+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9a380bc1e0 con 0x7f9a4c07b840 2024-05-09T08:27:41.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.363+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a4c073530 con 0x7f9a4c07b840 2024-05-09T08:27:41.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.364+0000 7f9a42a00640 1 -- 172.21.15.133:0/1728834614 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f9a380bbae0 con 0x7f9a4c07b840 2024-05-09T08:27:41.364 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:41.365 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-09T08:27:41.365 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2024-05-09T08:27:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 msgr2=0x7f9a300a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 0x7f9a300a51a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005d00 tx=0x7f9a3c005c70 comp rx=0 tx=0).stop 2024-05-09T08:27:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 msgr2=0x7f9a4c071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:41.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 0x7f9a4c071460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a3802a930 tx=0x7f9a38032c70 comp rx=0 tx=0).stop 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 shutdown_connections 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9a300a2ce0 0x7f9a300a51a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c07c1e0 0x7f9a4c0724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a4c07b840 0x7f9a4c071460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 --2- 172.21.15.133:0/1728834614 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c07a600 0x7f9a4c070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 >> 172.21.15.133:0/1728834614 conn(0x7f9a4c076e40 msgr2=0x7f9a4c07ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.367+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 shutdown_connections 2024-05-09T08:27:41.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:41.368+0000 7f9a50a00640 1 -- 172.21.15.133:0/1728834614 wait complete. 2024-05-09T08:27:41.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:41 smithi133 ceph-mon[29610]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:41.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:41 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1728834614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:42.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:41 smithi160 ceph-mon[34070]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:42.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:41 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1728834614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:42.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:41 smithi188 ceph-mon[34645]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:42.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:41 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1728834614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:42.427 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:27:42 smithi133 systemd[1]: Started Ceph mds.g for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:43.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: Deploying daemon mds.a on smithi133 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:43.557 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:boot 2024-05-09T08:27:43.558 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: fsmap 10 up:standby 2024-05-09T08:27:43.558 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:43 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:27:43.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 -- 172.21.15.133:0/1753980259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 msgr2=0x7fe53c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:43.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 --2- 172.21.15.133:0/1753980259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1097e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe534009900 tx=0x7fe53402a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 -- 172.21.15.133:0/1753980259 shutdown_connections 2024-05-09T08:27:43.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 --2- 172.21.15.133:0/1753980259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1097e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 --2- 172.21.15.133:0/1753980259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 --2- 172.21.15.133:0/1753980259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 -- 172.21.15.133:0/1753980259 >> 172.21.15.133:0/1753980259 conn(0x7fe53c0fbb00 msgr2=0x7fe53c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:43.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 -- 172.21.15.133:0/1753980259 shutdown_connections 2024-05-09T08:27:43.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.586+0000 7fe542400640 1 -- 172.21.15.133:0/1753980259 wait complete. 2024-05-09T08:27:43.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 Processor -- start 2024-05-09T08:27:43.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 -- start start 2024-05-09T08:27:43.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:43.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d330 con 0x7fe53c1073d0 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.587+0000 7fe542400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d1b0 con 0x7fe53c102170 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe542400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d4b0 con 0x7fe53c104aa0 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:36278/0 (socket says 172.21.15.133:36278) 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 -- 172.21.15.133:0/2582287785 learned_addr learned my addr 172.21.15.133:0/2582287785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541a00640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe53be00640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:43.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 -- 172.21.15.133:0/2582287785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 msgr2=0x7fe53c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 -- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 msgr2=0x7fe53c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe541000640 1 -- 172.21.15.133:0/2582287785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c1a17c0 con 0x7fe53c102170 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.588+0000 7fe53be00640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe541000640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe53000eab0 tx=0x7fe53000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53000ce40 con 0x7fe53c102170 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe530011950 con 0x7fe53c102170 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530010ae0 con 0x7fe53c102170 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c1a1ab0 con 0x7fe53c102170 2024-05-09T08:27:43.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.589+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1a1fb0 con 0x7fe53c102170 2024-05-09T08:27:43.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.591+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c068570 con 0x7fe53c102170 2024-05-09T08:27:43.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.592+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe530011b00 con 0x7fe53c102170 2024-05-09T08:27:43.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.593+0000 7fe539600640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 0x7fe5240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:43.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.593+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe5300cf280 con 0x7fe53c102170 2024-05-09T08:27:43.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.594+0000 7fe53be00640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 0x7fe5240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:43.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.594+0000 7fe53be00640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 0x7fe5240a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe53c19b0b0 tx=0x7fe528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:43.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.598+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe530016030 con 0x7fe53c102170 2024-05-09T08:27:43.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: Deploying daemon mds.a on smithi133 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:boot 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: fsmap 10 up:standby 2024-05-09T08:27:43.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:43 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: Deploying daemon mds.a on smithi133 2024-05-09T08:27:43.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:boot 2024-05-09T08:27:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: fsmap 10 up:standby 2024-05-09T08:27:43.684 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:43 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:27:43.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.848+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe53c062430 con 0x7fe53c102170 2024-05-09T08:27:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.849+0000 7fe539600640 1 -- 172.21.15.133:0/2582287785 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8848 (secure 0 0 0) 0x7fe53009c5e0 con 0x7fe53c102170 2024-05-09T08:27:43.850 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:43.850 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-05-09T08:27:43.850 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 11 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.852+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 msgr2=0x7fe5240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.852+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 0x7fe5240a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe53c19b0b0 tx=0x7fe528005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.852+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 msgr2=0x7fe53c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe53000eab0 tx=0x7fe53000ef70 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 shutdown_connections 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe5240a2b90 0x7fe5240a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe53c1073d0 0x7fe53c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe53c104aa0 0x7fe53c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 --2- 172.21.15.133:0/2582287785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe53c102170 0x7fe53c199ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:43.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 >> 172.21.15.133:0/2582287785 conn(0x7fe53c0fbb00 msgr2=0x7fe53c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:43.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 shutdown_connections 2024-05-09T08:27:43.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:43.853+0000 7fe542400640 1 -- 172.21.15.133:0/2582287785 wait complete. 2024-05-09T08:27:44.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:44 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2582287785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:44.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:44 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2582287785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:44 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2582287785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:45.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:45 smithi133 ceph-mon[29610]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:45.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:45 smithi160 ceph-mon[34070]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:45.678 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:45 smithi133 systemd[1]: Starting Ceph mds.a for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:45.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:45 smithi188 ceph-mon[34645]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:45.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:46.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 -- 172.21.15.133:0/3153736359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc109860 msgr2=0x7f45cc1100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:46.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 --2- 172.21.15.133:0/3153736359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc109860 0x7f45cc1100f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ca0 tx=0x7f45c402a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 -- 172.21.15.133:0/3153736359 shutdown_connections 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 --2- 172.21.15.133:0/3153736359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc109860 0x7f45cc1100f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 --2- 172.21.15.133:0/3153736359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 0x7f45cc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 --2- 172.21.15.133:0/3153736359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc107cb0 0x7f45cc1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 -- 172.21.15.133:0/3153736359 >> 172.21.15.133:0/3153736359 conn(0x7f45cc076c70 msgr2=0x7f45cc079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 -- 172.21.15.133:0/3153736359 shutdown_connections 2024-05-09T08:27:46.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.085+0000 7f45d2c00640 1 -- 172.21.15.133:0/3153736359 wait complete. 2024-05-09T08:27:46.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.086+0000 7f45d2c00640 1 Processor -- start 2024-05-09T08:27:46.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 -- start start 2024-05-09T08:27:46.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 0x7f45cc19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc109860 0x7f45cc1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113980 con 0x7f45cc107cb0 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113800 con 0x7f45cc109860 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc113b00 con 0x7f45cc108ea0 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60842/0 (socket says 172.21.15.133:60842) 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d1800640 1 -- 172.21.15.133:0/845185165 learned_addr learned my addr 172.21.15.133:0/845185165 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d0e00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 0x7f45cc19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.087+0000 7f45d2200640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc109860 0x7f45cc1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 -- 172.21.15.133:0/845185165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 msgr2=0x7f45cc19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 0x7f45cc19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 -- 172.21.15.133:0/845185165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc109860 msgr2=0x7f45cc1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc109860 0x7f45cc1a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 -- 172.21.15.133:0/845185165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc1a5b90 con 0x7f45cc107cb0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d1800640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9c0 tx=0x7f45c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c000cd80 con 0x7f45cc107cb0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c00035f0 con 0x7f45cc107cb0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0010690 con 0x7f45cc107cb0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.088+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc1a5e80 con 0x7f45cc107cb0 2024-05-09T08:27:46.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.089+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1a6290 con 0x7f45cc107cb0 2024-05-09T08:27:46.091 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:45 smithi133 podman[70937]: 2024-05-09 08:27:45.677689801 +0000 UTC m=+0.034953000 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:46.091 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:45 smithi133 podman[70937]: 2024-05-09 08:27:45.803367682 +0000 UTC m=+0.160630919 container create 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T08:27:46.091 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 podman[70937]: 2024-05-09 08:27:46.089728644 +0000 UTC m=+0.446991860 container init 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T08:27:46.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.090+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc10ef70 con 0x7f45cc107cb0 2024-05-09T08:27:46.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.092+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f45c00037a0 con 0x7f45cc107cb0 2024-05-09T08:27:46.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.095+0000 7f45ca000640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 0x7f45b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:46.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.096+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f45c00cefa0 con 0x7f45cc107cb0 2024-05-09T08:27:46.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.096+0000 7f45d0e00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 0x7f45b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:46.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.096+0000 7f45d0e00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 0x7f45b40a50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19f480 tx=0x7f45b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:46.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.101+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f45c009c300 con 0x7f45cc107cb0 2024-05-09T08:27:46.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.372+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45cc02d1c0 con 0x7f45cc107cb0 2024-05-09T08:27:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.373+0000 7f45ca000640 1 -- 172.21.15.133:0/845185165 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f45c009bc00 con 0x7f45cc107cb0 2024-05-09T08:27:46.373 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:46.373 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-09T08:27:46.373 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 12 2024-05-09T08:27:46.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 msgr2=0x7f45b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 0x7f45b40a50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19f480 tx=0x7f45b8005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 msgr2=0x7f45cc19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9c0 tx=0x7f45c000ee80 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 shutdown_connections 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f45b40a2c10 0x7f45b40a50d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45cc109860 0x7f45cc1a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45cc108ea0 0x7f45cc19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 --2- 172.21.15.133:0/845185165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45cc107cb0 0x7f45cc19dfc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.375+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 >> 172.21.15.133:0/845185165 conn(0x7f45cc076c70 msgr2=0x7f45cc10c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.376+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 shutdown_connections 2024-05-09T08:27:46.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:46.376+0000 7f45d2c00640 1 -- 172.21.15.133:0/845185165 wait complete. 2024-05-09T08:27:46.395 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 podman[70937]: 2024-05-09 08:27:46.099855275 +0000 UTC m=+0.457118471 container start 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:27:46.395 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 ceph-mds[70994]: -- 172.21.15.133:0/590013172 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ed7e49a40 con 0x558ed7e1c800 2024-05-09T08:27:46.395 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: starting mds.a at 2024-05-09T08:27:46.395 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 bash[70937]: 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec 2024-05-09T08:27:46.395 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:27:46 smithi133 systemd[1]: Started Ceph mds.a for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:boot 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: fsmap 11 up:standby 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:47.581 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: Deploying daemon mds.b on smithi160 2024-05-09T08:27:47.582 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/845185165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:47.582 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:47 smithi133 ceph-mon[29610]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:boot 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: fsmap 11 up:standby 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:47.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:47.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:47.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: Deploying daemon mds.b on smithi160 2024-05-09T08:27:47.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/845185165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:47.664 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:47 smithi160 ceph-mon[34070]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:47.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:boot 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: fsmap 11 up:standby 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: Deploying daemon mds.b on smithi160 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/845185165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:47.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:47 smithi188 ceph-mon[34645]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:48.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 -- 172.21.15.133:0/3345583046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 msgr2=0x7f9914105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 --2- 172.21.15.133:0/3345583046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f9914105100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009900 tx=0x7f98fc02a410 comp rx=0 tx=0).stop 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 -- 172.21.15.133:0/3345583046 shutdown_connections 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 --2- 172.21.15.133:0/3345583046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914105640 0x7f991410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 --2- 172.21.15.133:0/3345583046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f9914105100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 --2- 172.21.15.133:0/3345583046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914103a90 0x7f9914103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.607+0000 7f9919200640 1 -- 172.21.15.133:0/3345583046 >> 172.21.15.133:0/3345583046 conn(0x7f99140ff020 msgr2=0x7f9914101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:48.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.608+0000 7f9919200640 1 -- 172.21.15.133:0/3345583046 shutdown_connections 2024-05-09T08:27:48.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.608+0000 7f9919200640 1 -- 172.21.15.133:0/3345583046 wait complete. 2024-05-09T08:27:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 Processor -- start 2024-05-09T08:27:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 -- start start 2024-05-09T08:27:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 0x7f9914199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:48.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914105640 0x7f99141a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f991410f760 con 0x7f9914103a90 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f991410f5e0 con 0x7f9914104c80 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.609+0000 7f9919200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f991410f8e0 con 0x7f9914105640 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9909400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 0x7f9914199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9909400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:59762/0 (socket says 172.21.15.133:59762) 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9909400640 1 -- 172.21.15.133:0/2295640352 learned_addr learned my addr 172.21.15.133:0/2295640352 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9912800640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914105640 0x7f99141a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 -- 172.21.15.133:0/2295640352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914105640 msgr2=0x7f99141a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914105640 0x7f99141a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 -- 172.21.15.133:0/2295640352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 msgr2=0x7f991419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 -- 172.21.15.133:0/2295640352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99141a19d0 con 0x7f9914103a90 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9909400640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f9911e00640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 0x7f9914199dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9914104ae0 tx=0x7f990400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.610+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904017070 con 0x7f9914103a90 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.611+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904010460 con 0x7f9914103a90 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.611+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141a1cc0 con 0x7f9914103a90 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.611+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990400f620 con 0x7f9914103a90 2024-05-09T08:27:48.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.611+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a21c0 con 0x7f9914103a90 2024-05-09T08:27:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.612+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991410ad50 con 0x7f9914103a90 2024-05-09T08:27:48.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.614+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f990401a070 con 0x7f9914103a90 2024-05-09T08:27:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.615+0000 7f990b400640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 0x7f98f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.615+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f99040ce5a0 con 0x7f9914103a90 2024-05-09T08:27:48.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.615+0000 7f9909400640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 0x7f98f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:48.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.616+0000 7f9909400640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 0x7f98f80a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0096c0 tx=0x7f98fc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:48.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.620+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f990409b2b0 con 0x7f9914103a90 2024-05-09T08:27:48.663 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:48 smithi160 systemd[1]: Starting Ceph mds.b for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:27:48.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.860+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9914062460 con 0x7f9914103a90 2024-05-09T08:27:48.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.861+0000 7f990b400640 1 -- 172.21.15.133:0/2295640352 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f990409f5b0 con 0x7f9914103a90 2024-05-09T08:27:48.861 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:48.861 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-09T08:27:48.862 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 12 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 msgr2=0x7f98f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 0x7f98f80a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0096c0 tx=0x7f98fc0315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 msgr2=0x7f9914199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 0x7f9914199dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9914104ae0 tx=0x7f990400dbb0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 shutdown_connections 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f98f80a2b90 0x7f98f80a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9914105640 0x7f99141a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9914104c80 0x7f991419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 --2- 172.21.15.133:0/2295640352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9914103a90 0x7f9914199dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:48.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 >> 172.21.15.133:0/2295640352 conn(0x7f99140ff020 msgr2=0x7f9914107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 shutdown_connections 2024-05-09T08:27:48.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:48.864+0000 7f9919200640 1 -- 172.21.15.133:0/2295640352 wait complete. 2024-05-09T08:27:48.996 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:48 smithi160 podman[64447]: 2024-05-09 08:27:48.612104499 +0000 UTC m=+0.018113455 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:27:48.996 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:48 smithi160 podman[64447]: 2024-05-09 08:27:48.743860636 +0000 UTC m=+0.149869564 container create 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T08:27:48.997 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:48 smithi160 podman[64447]: 2024-05-09 08:27:48.997502556 +0000 UTC m=+0.403511485 container init 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T08:27:49.381 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:49 smithi160 podman[64447]: 2024-05-09 08:27:49.000209324 +0000 UTC m=+0.406218272 container start 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T08:27:49.381 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mds[64466]: -- 172.21.15.160:0/2653694267 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5615974d1a40 con 0x5615974a4800 2024-05-09T08:27:49.382 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: starting mds.b at 2024-05-09T08:27:49.382 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:49 smithi160 bash[64447]: 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 2024-05-09T08:27:49.382 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:49 smithi160 systemd[1]: Started Ceph mds.b for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2295640352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2295640352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2295640352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:50.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:50.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:27:50.980 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:50 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:boot 2024-05-09T08:27:50.980 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:50 smithi133 ceph-mon[29610]: fsmap 12 up:standby 2024-05-09T08:27:50.981 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:50 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:27:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 -- 172.21.15.133:0/1102262022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f08102a90 msgr2=0x7f0f08102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1102262022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f08102a90 0x7f0f08102f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009900 tx=0x7f0ef002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 -- 172.21.15.133:0/1102262022 shutdown_connections 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1102262022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 0x7f0f08109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1102262022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f08102a90 0x7f0f08102f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1102262022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f081018a0 0x7f0f08101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 -- 172.21.15.133:0/1102262022 >> 172.21.15.133:0/1102262022 conn(0x7f0f080fce10 msgr2=0x7f0f080ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 -- 172.21.15.133:0/1102262022 shutdown_connections 2024-05-09T08:27:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.081+0000 7f0f0f200640 1 -- 172.21.15.133:0/1102262022 wait complete. 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.082+0000 7f0f0f200640 1 Processor -- start 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.082+0000 7f0f0f200640 1 -- start start 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f08102a90 0x7f0f0819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 0x7f0f081a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810d570 con 0x7f0f08102a90 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810d3f0 con 0x7f0f08103450 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810d6f0 con 0x7f0f081018a0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:38768/0 (socket says 172.21.15.133:38768) 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f08102a90 0x7f0f0819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 -- 172.21.15.133:0/1787832663 learned_addr learned my addr 172.21.15.133:0/1787832663 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f0d200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 0x7f0f081a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 -- 172.21.15.133:0/1787832663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 msgr2=0x7f0f081a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 0x7f0f081a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 -- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f08102a90 msgr2=0x7f0f0819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f08102a90 0x7f0f0819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.083+0000 7f0f07e00640 1 -- 172.21.15.133:0/1787832663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081a19f0 con 0x7f0f081018a0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f07e00640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000ebc0 tx=0x7f0f0000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00003b70 con 0x7f0f081018a0 2024-05-09T08:27:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f00003d10 con 0x7f0f081018a0 2024-05-09T08:27:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f081a1ce0 con 0x7f0f081018a0 2024-05-09T08:27:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00019980 con 0x7f0f081018a0 2024-05-09T08:27:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.084+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a21a0 con 0x7f0f081018a0 2024-05-09T08:27:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.085+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f081a2430 con 0x7f0f081018a0 2024-05-09T08:27:51.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.087+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0f00004b60 con 0x7f0f081018a0 2024-05-09T08:27:51.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.088+0000 7f0f04c00640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 0x7f0eec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.089+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0f000cf340 con 0x7f0f081018a0 2024-05-09T08:27:51.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.089+0000 7f0f07400640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 0x7f0eec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.089+0000 7f0f07400640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 0x7f0eec0a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0031440 tx=0x7f0ef00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.091+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0f0009c6a0 con 0x7f0f081018a0 2024-05-09T08:27:51.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:50 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:boot 2024-05-09T08:27:51.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:50 smithi160 ceph-mon[34070]: fsmap 12 up:standby 2024-05-09T08:27:51.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:50 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:27:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:50 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:boot 2024-05-09T08:27:51.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:50 smithi188 ceph-mon[34645]: fsmap 12 up:standby 2024-05-09T08:27:51.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:50 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:27:51.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.344+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f081a2920 con 0x7f0f081018a0 2024-05-09T08:27:51.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.346+0000 7f0f04c00640 1 -- 172.21.15.133:0/1787832663 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10503 (secure 0 0 0) 0x7f0f0009bfa0 con 0x7f0f081018a0 2024-05-09T08:27:51.346 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:51.346 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24536,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24556,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-09T08:27:51.346 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 13 2024-05-09T08:27:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 msgr2=0x7f0eec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 0x7f0eec0a50d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0031440 tx=0x7f0ef00315b0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 msgr2=0x7f0f08199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000ebc0 tx=0x7f0f0000c6a0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 shutdown_connections 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0eec0a2c10 0x7f0eec0a50d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f08103450 0x7f0f081a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f08102a90 0x7f0f0819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 --2- 172.21.15.133:0/1787832663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f081018a0 0x7f0f08199dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 >> 172.21.15.133:0/1787832663 conn(0x7f0f080fce10 msgr2=0x7f0f08073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 shutdown_connections 2024-05-09T08:27:51.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.348+0000 7f0f0f200640 1 -- 172.21.15.133:0/1787832663 wait complete. 2024-05-09T08:27:51.367 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:27:51.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 -- 172.21.15.133:0/810640755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 msgr2=0x7f94b81046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 --2- 172.21.15.133:0/810640755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b81046f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802a410 comp rx=0 tx=0).stop 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 -- 172.21.15.133:0/810640755 shutdown_connections 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 --2- 172.21.15.133:0/810640755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 --2- 172.21.15.133:0/810640755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 0x7f94b8107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 --2- 172.21.15.133:0/810640755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b81046f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 -- 172.21.15.133:0/810640755 >> 172.21.15.133:0/810640755 conn(0x7f94b80fbd10 msgr2=0x7f94b80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 -- 172.21.15.133:0/810640755 shutdown_connections 2024-05-09T08:27:51.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.550+0000 7f94bd800640 1 -- 172.21.15.133:0/810640755 wait complete. 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 Processor -- start 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 -- start start 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 0x7f94b819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810d530 con 0x7f94b8102300 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810d3b0 con 0x7f94b8107560 2024-05-09T08:27:51.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94bd800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94b810d6b0 con 0x7f94b8104c30 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94b6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.551+0000 7f94b6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60902/0 (socket says 172.21.15.133:60902) 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 -- 172.21.15.133:0/1287167404 learned_addr learned my addr 172.21.15.133:0/1287167404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b7400640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6000640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 0x7f94b819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 -- 172.21.15.133:0/1287167404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 msgr2=0x7f94b819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 0x7f94b819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 -- 172.21.15.133:0/1287167404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 msgr2=0x7f94b81a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 -- 172.21.15.133:0/1287167404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b81a1a10 con 0x7f94b8102300 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b7400640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.552+0000 7f94b6a00640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009230 tx=0x7f94a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.553+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a803f030 con 0x7f94b8102300 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.553+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a8003ce0 con 0x7f94b8102300 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.553+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8035070 con 0x7f94b8102300 2024-05-09T08:27:51.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.553+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b81a1ca0 con 0x7f94b8102300 2024-05-09T08:27:51.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.553+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b81a2140 con 0x7f94b8102300 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.554+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b81090b0 con 0x7f94b8102300 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.556+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f94a8042030 con 0x7f94b8102300 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.557+0000 7f94af400640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 0x7f949c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.557+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f94a80eeff0 con 0x7f94b8102300 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.558+0000 7f94b6000640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 0x7f949c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.558+0000 7f94b6000640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 0x7f949c0a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94b819b300 tx=0x7f94a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.561+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f94a80bbd20 con 0x7f94b8102300 2024-05-09T08:27:51.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.776+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94b80624d0 con 0x7f94b8102300 2024-05-09T08:27:51.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.778+0000 7f94af400640 1 -- 172.21.15.133:0/1287167404 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18524 (secure 0 0 0) 0x7f94a80c0020 con 0x7f94b8102300 2024-05-09T08:27:51.778 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:51.779 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":82,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:00.078249+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 msgr2=0x7f949c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 0x7f949c0a50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94b819b300 tx=0x7f94a4005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 msgr2=0x7f94b8199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009230 tx=0x7f94a80042f0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 shutdown_connections 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f949c0a2c10 0x7f949c0a50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94b8107560 0x7f94b81a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94b8104c30 0x7f94b819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 --2- 172.21.15.133:0/1287167404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94b8102300 0x7f94b8199e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 >> 172.21.15.133:0/1287167404 conn(0x7f94b80fbd10 msgr2=0x7f94b8100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 shutdown_connections 2024-05-09T08:27:51.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.781+0000 7f94bd800640 1 -- 172.21.15.133:0/1287167404 wait complete. 2024-05-09T08:27:51.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:27:51.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:27:51.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 -- 172.21.15.133:0/3746018621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 msgr2=0x7f281810ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 --2- 172.21.15.133:0/3746018621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f281810ba70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2810009be0 tx=0x7f281002a750 comp rx=0 tx=0).stop 2024-05-09T08:27:51.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 -- 172.21.15.133:0/3746018621 shutdown_connections 2024-05-09T08:27:51.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 --2- 172.21.15.133:0/3746018621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f281810ba70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 --2- 172.21.15.133:0/3746018621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818074690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 --2- 172.21.15.133:0/3746018621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 0x7f2818073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 -- 172.21.15.133:0/3746018621 >> 172.21.15.133:0/3746018621 conn(0x7f28180fccd0 msgr2=0x7f28180ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:51.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 -- 172.21.15.133:0/3746018621 shutdown_connections 2024-05-09T08:27:51.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.976+0000 7f281e200640 1 -- 172.21.15.133:0/3746018621 wait complete. 2024-05-09T08:27:51.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 Processor -- start 2024-05-09T08:27:51.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 -- start start 2024-05-09T08:27:51.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 0x7f281819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f28181a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f281810f5c0 con 0x7f2818074230 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f281810f440 con 0x7f2818075810 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.977+0000 7f281e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f281810f740 con 0x7f28181095d0 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60906/0 (socket says 172.21.15.133:60906) 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 -- 172.21.15.133:0/1723319336 learned_addr learned my addr 172.21.15.133:0/1723319336 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281d800640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f28181a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f2817e00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 0x7f281819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 -- 172.21.15.133:0/1723319336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 msgr2=0x7f28181a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f28181a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 -- 172.21.15.133:0/1723319336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 msgr2=0x7f281819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 0x7f281819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 -- 172.21.15.133:0/1723319336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a17f0 con 0x7f2818074230 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.978+0000 7f281ce00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9f0 tx=0x7f280c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.979+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c00cd80 con 0x7f2818074230 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.979+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c005500 con 0x7f2818074230 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.979+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a1ae0 con 0x7f2818074230 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.979+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c010690 con 0x7f2818074230 2024-05-09T08:27:51.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.979+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181a1fe0 con 0x7f2818074230 2024-05-09T08:27:51.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.980+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2818108390 con 0x7f2818074230 2024-05-09T08:27:51.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.982+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f280c0046c0 con 0x7f2818074230 2024-05-09T08:27:51.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.983+0000 7f2815600640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:51.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.984+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f280c0cef80 con 0x7f2818074230 2024-05-09T08:27:51.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.984+0000 7f2817e00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:51.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.984+0000 7f2817e00640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 0x7f28000a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f281819b0e0 tx=0x7f2804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:51.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:51.988+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f280c09c380 con 0x7f2818074230 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1787832663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:52.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:52.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1787832663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:52.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1787832663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:27:52.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.185+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f281802d1c0 con 0x7f2818074230 2024-05-09T08:27:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.186+0000 7f2815600640 1 -- 172.21.15.133:0/1723319336 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f280c09bc80 con 0x7f2818074230 2024-05-09T08:27:52.186 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:27:52.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 msgr2=0x7f28000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 0x7f28000a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f281819b0e0 tx=0x7f2804009260 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 msgr2=0x7f2818199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f280c00e9f0 tx=0x7f280c00edf0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 shutdown_connections 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f28000a2b90 0x7f28000a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28181095d0 0x7f28181a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2818075810 0x7f281819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 --2- 172.21.15.133:0/1723319336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818074230 0x7f2818199bd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.188+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 >> 172.21.15.133:0/1723319336 conn(0x7f28180fccd0 msgr2=0x7f2818105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.189+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 shutdown_connections 2024-05-09T08:27:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.189+0000 7f281e200640 1 -- 172.21.15.133:0/1723319336 wait complete. 2024-05-09T08:27:52.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T08:27:52.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 -- 172.21.15.133:0/65764606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 msgr2=0x7f84bc109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 --2- 172.21.15.133:0/65764606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc109b10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009860 tx=0x7f84b402a340 comp rx=0 tx=0).stop 2024-05-09T08:27:52.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 -- 172.21.15.133:0/65764606 shutdown_connections 2024-05-09T08:27:52.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 --2- 172.21.15.133:0/65764606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc109b10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 --2- 172.21.15.133:0/65764606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1028c0 0x7f84bc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 --2- 172.21.15.133:0/65764606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1016d0 0x7f84bc101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.383+0000 7f84bbe00640 1 -- 172.21.15.133:0/65764606 >> 172.21.15.133:0/65764606 conn(0x7f84bc0fcc40 msgr2=0x7f84bc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.384+0000 7f84bbe00640 1 -- 172.21.15.133:0/65764606 shutdown_connections 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.384+0000 7f84bbe00640 1 -- 172.21.15.133:0/65764606 wait complete. 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.384+0000 7f84bbe00640 1 Processor -- start 2024-05-09T08:27:52.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 -- start start 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1016d0 0x7f84bc199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d350 con 0x7f84bc1028c0 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d1d0 con 0x7f84bc1016d0 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10d4d0 con 0x7f84bc103280 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60918/0 (socket says 172.21.15.133:60918) 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 -- 172.21.15.133:0/3184267766 learned_addr learned my addr 172.21.15.133:0/3184267766 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84baa00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1016d0 0x7f84bc199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84bb400640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 -- 172.21.15.133:0/3184267766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 msgr2=0x7f84bc1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 -- 172.21.15.133:0/3184267766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1016d0 msgr2=0x7f84bc199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.385+0000 7f84b2000640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1016d0 0x7f84bc199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84b2000640 1 -- 172.21.15.133:0/3184267766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc1a1800 con 0x7f84bc1028c0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84b2000640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9f0 tx=0x7f84a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400cd80 con 0x7f84bc1028c0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc1a1af0 con 0x7f84bc1028c0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a4005500 con 0x7f84bc1028c0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4010690 con 0x7f84bc1028c0 2024-05-09T08:27:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.386+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a1ff0 con 0x7f84bc1028c0 2024-05-09T08:27:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.387+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84bc1a2280 con 0x7f84bc1028c0 2024-05-09T08:27:52.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.389+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f84a40046c0 con 0x7f84bc1028c0 2024-05-09T08:27:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.391+0000 7f84b3e00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.391+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f84a40cefe0 con 0x7f84bc1028c0 2024-05-09T08:27:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.391+0000 7f84baa00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.392+0000 7f84baa00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0098d0 tx=0x7f84ac010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:52.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.395+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84a409c380 con 0x7f84bc1028c0 2024-05-09T08:27:52.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.600+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f84bc1a2450 con 0x7f84bc1028c0 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.601+0000 7f84b3e00640 1 -- 172.21.15.133:0/3184267766 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f84a409bc80 con 0x7f84bc1028c0 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-05-09T08:27:52.601 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-05-09T08:27:52.602 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:27:52.602 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.603+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 msgr2=0x7f84a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.603+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0098d0 tx=0x7f84ac010040 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.603+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 msgr2=0x7f84bc19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.603+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9f0 tx=0x7f84a400edf0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 shutdown_connections 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84bc103280 0x7f84bc1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc1028c0 0x7f84bc19a170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 --2- 172.21.15.133:0/3184267766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84bc1016d0 0x7f84bc199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 >> 172.21.15.133:0/3184267766 conn(0x7f84bc0fcc40 msgr2=0x7f84bc073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:52.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 shutdown_connections 2024-05-09T08:27:52.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.604+0000 7f84bbe00640 1 -- 172.21.15.133:0/3184267766 wait complete. 2024-05-09T08:27:52.623 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T08:27:52.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 -- 172.21.15.133:0/1883902988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 msgr2=0x7f83381070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 --2- 172.21.15.133:0/1883902988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f83381070b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8324009860 tx=0x7f832402a340 comp rx=0 tx=0).stop 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 -- 172.21.15.133:0/1883902988 shutdown_connections 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 --2- 172.21.15.133:0/1883902988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 0x7f83381099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 --2- 172.21.15.133:0/1883902988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f83381070b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 --2- 172.21.15.133:0/1883902988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 -- 172.21.15.133:0/1883902988 >> 172.21.15.133:0/1883902988 conn(0x7f83380fbce0 msgr2=0x7f83380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 -- 172.21.15.133:0/1883902988 shutdown_connections 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.803+0000 7f8340600640 1 -- 172.21.15.133:0/1883902988 wait complete. 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.804+0000 7f8340600640 1 Processor -- start 2024-05-09T08:27:52.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.804+0000 7f8340600640 1 -- start start 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 0x7f83381a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d4c0 con 0x7f83381075f0 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d340 con 0x7f8338104cc0 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f8340600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d640 con 0x7f8338102390 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:38840/0 (socket says 172.21.15.133:38840) 2024-05-09T08:27:52.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833d200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:59842/0 (socket says 172.21.15.133:59842) 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833dc00640 1 -- 172.21.15.133:0/1499574703 learned_addr learned my addr 172.21.15.133:0/1499574703 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.805+0000 7f833e600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 0x7f83381a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 -- 172.21.15.133:0/1499574703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 msgr2=0x7f833819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 -- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 msgr2=0x7f83381a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 0x7f83381a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 -- 172.21.15.133:0/1499574703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83381a1a60 con 0x7f8338102390 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833d200640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f833dc00640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e9c0 tx=0x7f832c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00cde0 con 0x7f8338102390 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c0035f0 con 0x7f8338102390 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83381a1d50 con 0x7f8338102390 2024-05-09T08:27:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c010650 con 0x7f8338102390 2024-05-09T08:27:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.806+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a21c0 con 0x7f8338102390 2024-05-09T08:27:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.807+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83381090e0 con 0x7f8338102390 2024-05-09T08:27:52.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.809+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f832c004020 con 0x7f8338102390 2024-05-09T08:27:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.811+0000 7f8336000640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 0x7f83200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.811+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f832c0cef00 con 0x7f8338102390 2024-05-09T08:27:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.811+0000 7f833d200640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 0x7f83200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.811+0000 7f833d200640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 0x7f83200a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8324030410 tx=0x7f8324030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:52.814+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f832c09c260 con 0x7f8338102390 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1287167404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1723319336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:52.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:52.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:52.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:52.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:52 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3184267766' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T08:27:53.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.017+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f833802d1c0 con 0x7f8338102390 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1287167404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1723319336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:53.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:53.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:52 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3184267766' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1287167404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1723319336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:27:53.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3184267766' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T08:27:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.750+0000 7f8336000640 1 -- 172.21.15.133:0/1499574703 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f833802d1c0 con 0x7f8338102390 2024-05-09T08:27:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-05-09T08:27:53.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.760+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 msgr2=0x7f83200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:53.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.760+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 0x7f83200a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8324030410 tx=0x7f8324030550 comp rx=0 tx=0).stop 2024-05-09T08:27:53.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.760+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 msgr2=0x7f8338199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:53.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.760+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f832c00e9c0 tx=0x7f832c00ee80 comp rx=0 tx=0).stop 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 shutdown_connections 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2b90 0x7f83200a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83381075f0 0x7f83381a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8338104cc0 0x7f833819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 --2- 172.21.15.133:0/1499574703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8338102390 0x7f8338199e40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 >> 172.21.15.133:0/1499574703 conn(0x7f83380fbce0 msgr2=0x7f8338101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 shutdown_connections 2024-05-09T08:27:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.761+0000 7f8340600640 1 -- 172.21.15.133:0/1499574703 wait complete. 2024-05-09T08:27:53.782 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T08:27:53.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 -- 172.21.15.133:0/2874241155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414103ab0 msgr2=0x7f4414103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 --2- 172.21.15.133:0/2874241155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414103ab0 0x7f4414103ed0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4404009ca0 tx=0x7f440402a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 -- 172.21.15.133:0/2874241155 shutdown_connections 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 --2- 172.21.15.133:0/2874241155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 0x7f441410bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 --2- 172.21.15.133:0/2874241155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414104ca0 0x7f4414105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 --2- 172.21.15.133:0/2874241155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414103ab0 0x7f4414103ed0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.962+0000 7f441a200640 1 -- 172.21.15.133:0/2874241155 >> 172.21.15.133:0/2874241155 conn(0x7f44140ff020 msgr2=0x7f4414101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.963+0000 7f441a200640 1 -- 172.21.15.133:0/2874241155 shutdown_connections 2024-05-09T08:27:53.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.963+0000 7f441a200640 1 -- 172.21.15.133:0/2874241155 wait complete. 2024-05-09T08:27:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.964+0000 7f441a200640 1 Processor -- start 2024-05-09T08:27:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.964+0000 7f441a200640 1 -- start start 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414104ca0 0x7f441419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 0x7f44141a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f441410f700 con 0x7f4414104ca0 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f441410f580 con 0x7f4414103ab0 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f441a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f441410f880 con 0x7f4414105660 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34020/0 (socket says 172.21.15.133:34020) 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 -- 172.21.15.133:0/3604209631 learned_addr learned my addr 172.21.15.133:0/3604209631 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:53.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4412a00640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414104ca0 0x7f441419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413e00640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 0x7f44141a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 -- 172.21.15.133:0/3604209631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 msgr2=0x7f44141a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 0x7f44141a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 -- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414104ca0 msgr2=0x7f441419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414104ca0 0x7f441419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.965+0000 7f4413400640 1 -- 172.21.15.133:0/3604209631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141a1a40 con 0x7f4414103ab0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f4413400640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4404009bf0 tx=0x7f4404031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440402fa20 con 0x7f4414103ab0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440402fbc0 con 0x7f4414103ab0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040466e0 con 0x7f4414103ab0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a1cd0 con 0x7f4414103ab0 2024-05-09T08:27:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.966+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a21d0 con 0x7f4414103ab0 2024-05-09T08:27:53.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.967+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441410ae70 con 0x7f4414103ab0 2024-05-09T08:27:53.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.969+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f440402fd70 con 0x7f4414103ab0 2024-05-09T08:27:53.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.970+0000 7f440be00640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 0x7f43f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:53.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.970+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f44040f00c0 con 0x7f4414103ab0 2024-05-09T08:27:53.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.970+0000 7f4412a00640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 0x7f43f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.971+0000 7f4412a00640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 0x7f43f80a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44000095f0 tx=0x7f4400005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:53.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:53.976+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f44040bd2b0 con 0x7f4414103ab0 2024-05-09T08:27:54.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:53 smithi160 ceph-mon[34070]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:54.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:53 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1499574703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:53 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.164+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f44140624d0 con 0x7f4414103ab0 2024-05-09T08:27:54.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:53 smithi133 ceph-mon[29610]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:54.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:53 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1499574703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:53 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:53 smithi188 ceph-mon[34645]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:53 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1499574703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:53 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T08:27:54.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.757+0000 7f440be00640 1 -- 172.21.15.133:0/3604209631 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f44040bcbb0 con 0x7f4414103ab0 2024-05-09T08:27:54.757 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-05-09T08:27:54.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.765+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 msgr2=0x7f43f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:54.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.765+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 0x7f43f80a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f44000095f0 tx=0x7f4400005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:54.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.765+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 msgr2=0x7f4414199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:54.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.765+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4404009bf0 tx=0x7f4404031ee0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 shutdown_connections 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43f80a2b90 0x7f43f80a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4414105660 0x7f44141a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414104ca0 0x7f441419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 --2- 172.21.15.133:0/3604209631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4414103ab0 0x7f4414199e20 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4404009bf0 tx=0x7f4404031ee0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 >> 172.21.15.133:0/3604209631 conn(0x7f44140ff020 msgr2=0x7f4414108040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:54.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.766+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 shutdown_connections 2024-05-09T08:27:54.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.768+0000 7f441a200640 1 -- 172.21.15.133:0/3604209631 wait complete. 2024-05-09T08:27:54.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 -- 172.21.15.133:0/1215517822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188102950 msgr2=0x7fe188102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 --2- 172.21.15.133:0/1215517822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188102950 0x7fe188102dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe174009900 tx=0x7fe17402a410 comp rx=0 tx=0).stop 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 -- 172.21.15.133:0/1215517822 shutdown_connections 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 --2- 172.21.15.133:0/1215517822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188103310 0x7fe188109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 --2- 172.21.15.133:0/1215517822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188102950 0x7fe188102dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 --2- 172.21.15.133:0/1215517822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 -- 172.21.15.133:0/1215517822 >> 172.21.15.133:0/1215517822 conn(0x7fe1880fccd0 msgr2=0x7fe1880ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 -- 172.21.15.133:0/1215517822 shutdown_connections 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.981+0000 7fe187e00640 1 -- 172.21.15.133:0/1215517822 wait complete. 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 Processor -- start 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 -- start start 2024-05-09T08:27:54.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188102950 0x7fe18819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d430 con 0x7fe188102950 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d2b0 con 0x7fe188101760 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.982+0000 7fe187e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe18810d5b0 con 0x7fe188103310 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe186000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188102950 0x7fe18819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe186a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:55288/0 (socket says 172.21.15.133:55288) 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe186a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34026/0 (socket says 172.21.15.133:34026) 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 -- 172.21.15.133:0/460541848 learned_addr learned my addr 172.21.15.133:0/460541848 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 -- 172.21.15.133:0/460541848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 msgr2=0x7fe188199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 -- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188102950 msgr2=0x7fe18819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188102950 0x7fe18819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 -- 172.21.15.133:0/460541848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881a18f0 con 0x7fe188103310 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe186a00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.983+0000 7fe187400640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe18000eb10 tx=0x7fe18000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.984+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180004b20 con 0x7fe188103310 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.984+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe180004cc0 con 0x7fe188103310 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.984+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a1be0 con 0x7fe188103310 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.984+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180010760 con 0x7fe188103310 2024-05-09T08:27:54.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.984+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a2100 con 0x7fe188103310 2024-05-09T08:27:54.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.985+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1881a2330 con 0x7fe188103310 2024-05-09T08:27:54.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.987+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe180005020 con 0x7fe188103310 2024-05-09T08:27:54.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.989+0000 7fe17f400640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 0x7fe16c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:54.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.989+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fe1800cf460 con 0x7fe188103310 2024-05-09T08:27:54.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.989+0000 7fe186a00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 0x7fe16c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:54.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.990+0000 7fe186a00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 0x7fe16c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe18819ab80 tx=0x7fe178005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:54.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:54.992+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe18009c4d0 con 0x7fe188103310 2024-05-09T08:27:55.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:54 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T08:27:55.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:54 smithi160 ceph-mon[34070]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T08:27:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:54 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T08:27:55.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:54 smithi133 ceph-mon[29610]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T08:27:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:54 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T08:27:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:54 smithi188 ceph-mon[34645]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T08:27:55.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:55.238+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe1881a2500 con 0x7fe188103310 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3604209631' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: pgmap v312: 33 pgs: 17 creating+peering, 15 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/460541848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:55 smithi160 ceph-mon[34070]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3604209631' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: pgmap v312: 33 pgs: 17 creating+peering, 15 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:56.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/460541848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:55 smithi133 ceph-mon[29610]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T08:27:56.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3604209631' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: pgmap v312: 33 pgs: 17 creating+peering, 15 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/460541848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T08:27:56.184 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:55 smithi188 ceph-mon[34645]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T08:27:56.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.220+0000 7fe17f400640 1 -- 172.21.15.133:0/460541848 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe18009bdd0 con 0x7fe188103310 2024-05-09T08:27:56.220 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 msgr2=0x7fe16c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 0x7fe16c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe18819ab80 tx=0x7fe178005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 msgr2=0x7fe1881a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe18000eb10 tx=0x7fe18000c2d0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 shutdown_connections 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe16c0a2c10 0x7fe16c0a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe188103310 0x7fe1881a1250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe188102950 0x7fe18819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 --2- 172.21.15.133:0/460541848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe188101760 0x7fe188199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.225+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 >> 172.21.15.133:0/460541848 conn(0x7fe1880fccd0 msgr2=0x7fe188073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:56.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.226+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 shutdown_connections 2024-05-09T08:27:56.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.227+0000 7fe187e00640 1 -- 172.21.15.133:0/460541848 wait complete. 2024-05-09T08:27:56.251 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:27:56.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 -- 172.21.15.133:0/2837766511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c0761f0 msgr2=0x7f1a2c076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 --2- 172.21.15.133:0/2837766511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c076610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009900 tx=0x7f1a1402a410 comp rx=0 tx=0).stop 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 -- 172.21.15.133:0/2837766511 shutdown_connections 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 --2- 172.21.15.133:0/2837766511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c077dd0 0x7f1a2c078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 --2- 172.21.15.133:0/2837766511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077430 0x7f1a2c077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 --2- 172.21.15.133:0/2837766511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c076610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 -- 172.21.15.133:0/2837766511 >> 172.21.15.133:0/2837766511 conn(0x7f1a2c0ff020 msgr2=0x7f1a2c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 -- 172.21.15.133:0/2837766511 shutdown_connections 2024-05-09T08:27:56.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.444+0000 7f1a33c00640 1 -- 172.21.15.133:0/2837766511 wait complete. 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.445+0000 7f1a33c00640 1 Processor -- start 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.445+0000 7f1a33c00640 1 -- start start 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c077430 0x7f1a2c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077dd0 0x7f1a2c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c113b00 con 0x7f1a2c077430 2024-05-09T08:27:56.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c113980 con 0x7f1a2c0761f0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a33c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c113c80 con 0x7f1a2c077dd0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c077430 0x7f1a2c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34044/0 (socket says 172.21.15.133:34044) 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 -- 172.21.15.133:0/649349846 learned_addr learned my addr 172.21.15.133:0/649349846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077dd0 0x7f1a2c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 -- 172.21.15.133:0/649349846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077dd0 msgr2=0x7f1a2c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077dd0 0x7f1a2c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 -- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c077430 msgr2=0x7f1a2c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c077430 0x7f1a2c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.446+0000 7f1a31200640 1 -- 172.21.15.133:0/649349846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c1a5dc0 con 0x7f1a2c0761f0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a31200640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009230 tx=0x7f1a140042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:56.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1403f030 con 0x7f1a2c0761f0 2024-05-09T08:27:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c1a6050 con 0x7f1a2c0761f0 2024-05-09T08:27:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a14003ce0 con 0x7f1a2c0761f0 2024-05-09T08:27:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14035070 con 0x7f1a2c0761f0 2024-05-09T08:27:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.447+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c1a6540 con 0x7f1a2c0761f0 2024-05-09T08:27:56.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.448+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c1105e0 con 0x7f1a2c0761f0 2024-05-09T08:27:56.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.450+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1a14042030 con 0x7f1a2c0761f0 2024-05-09T08:27:56.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.452+0000 7f1a2a000640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 0x7f1a0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.452+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1a140ef850 con 0x7f1a2c0761f0 2024-05-09T08:27:56.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.452+0000 7f1a23e00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 0x7f1a0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.453+0000 7f1a23e00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 0x7f1a0c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c19f6b0 tx=0x7f1a1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:56.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.454+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1a140bc210 con 0x7f1a2c0761f0 2024-05-09T08:27:56.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.652+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a2c062430 con 0x7f1a2c0761f0 2024-05-09T08:27:56.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.654+0000 7f1a2a000640 1 -- 172.21.15.133:0/649349846 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22300 (secure 0 0 0) 0x7f1a140c0510 con 0x7f1a2c0761f0 2024-05-09T08:27:56.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:56.655 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":85,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:55.240487+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:27:56.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.657+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 msgr2=0x7f1a0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.657+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 0x7f1a0c0a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c19f6b0 tx=0x7f1a1c005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:56.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.657+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 msgr2=0x7f1a2c19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.657+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009230 tx=0x7f1a140042f0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.657+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 shutdown_connections 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1a0c0a2b90 0x7f1a0c0a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a2c077dd0 0x7f1a2c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a2c077430 0x7f1a2c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 --2- 172.21.15.133:0/649349846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a2c0761f0 0x7f1a2c19e1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 >> 172.21.15.133:0/649349846 conn(0x7f1a2c0ff020 msgr2=0x7f1a2c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:56.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 shutdown_connections 2024-05-09T08:27:56.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.658+0000 7f1a33c00640 1 -- 172.21.15.133:0/649349846 wait complete. 2024-05-09T08:27:56.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:27:56.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 -- 172.21.15.133:0/2208281419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 msgr2=0x7fb2c0103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/2208281419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0103cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009860 tx=0x7fb2b402a340 comp rx=0 tx=0).stop 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 -- 172.21.15.133:0/2208281419 shutdown_connections 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/2208281419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0105460 0x7fb2c010bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/2208281419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0104aa0 0x7fb2c0104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/2208281419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0103cd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 -- 172.21.15.133:0/2208281419 >> 172.21.15.133:0/2208281419 conn(0x7fb2c00fee40 msgr2=0x7fb2c0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 -- 172.21.15.133:0/2208281419 shutdown_connections 2024-05-09T08:27:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.871+0000 7fb2c6a00640 1 -- 172.21.15.133:0/2208281419 wait complete. 2024-05-09T08:27:56.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.872+0000 7fb2c6a00640 1 Processor -- start 2024-05-09T08:27:56.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.872+0000 7fb2c6a00640 1 -- start start 2024-05-09T08:27:56.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 0x7fb2c019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010f530 con 0x7fb2c0105460 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010f3b0 con 0x7fb2c0104aa0 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c010f6b0 con 0x7fb2c01038b0 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33186/0 (socket says 172.21.15.133:33186) 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c6000640 1 -- 172.21.15.133:0/3413617522 learned_addr learned my addr 172.21.15.133:0/3413617522 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c5600640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.873+0000 7fb2c4c00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 0x7fb2c019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 -- 172.21.15.133:0/3413617522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 msgr2=0x7fb2c019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 0x7fb2c019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 -- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 msgr2=0x7fb2c01a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:56.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 -- 172.21.15.133:0/3413617522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c01a17d0 con 0x7fb2c01038b0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c6000640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c4c00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 0x7fb2c019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c5600640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0199bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402a860 tx=0x7fb2b4033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4037040 con 0x7fb2c01038b0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4030850 con 0x7fb2c01038b0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c01a1a60 con 0x7fb2c01038b0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4004920 con 0x7fb2c01038b0 2024-05-09T08:27:56.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.874+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c01a1f70 con 0x7fb2c01038b0 2024-05-09T08:27:56.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.875+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c010abf0 con 0x7fb2c01038b0 2024-05-09T08:27:56.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.877+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb2b4004050 con 0x7fb2c01038b0 2024-05-09T08:27:56.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.878+0000 7fb2be000640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 0x7fb2a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:56.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.879+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb2b40eecb0 con 0x7fb2c01038b0 2024-05-09T08:27:56.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.879+0000 7fb2c4c00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 0x7fb2a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:56.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.879+0000 7fb2c4c00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 0x7fb2a80a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0096c0 tx=0x7fb2ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:56.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:56.883+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb2b40bbcc0 con 0x7fb2c01038b0 2024-05-09T08:27:56.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T08:27:56.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T08:27:56.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T08:27:56.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T08:27:56.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/649349846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:56.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:56 smithi133 ceph-mon[29610]: pgmap v315: 97 pgs: 17 creating+peering, 55 unknown, 25 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T08:27:57.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.087+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2c0062430 con 0x7fb2c01038b0 2024-05-09T08:27:57.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.089+0000 7fb2be000640 1 -- 172.21.15.133:0/3413617522 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22287 (secure 0 0 0) 0x7fb2b40bb5c0 con 0x7fb2c01038b0 2024-05-09T08:27:57.089 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:27:57.089 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":86,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:56.775532+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 msgr2=0x7fb2a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 0x7fb2a80a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0096c0 tx=0x7fb2ac009260 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 msgr2=0x7fb2c0199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0199bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402a860 tx=0x7fb2b4033d90 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 shutdown_connections 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2a80a2c10 0x7fb2a80a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2c0105460 0x7fb2c01a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c0104aa0 0x7fb2c019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 --2- 172.21.15.133:0/3413617522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2c01038b0 0x7fb2c0199bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 >> 172.21.15.133:0/3413617522 conn(0x7fb2c00fee40 msgr2=0x7fb2c0107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:57.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 shutdown_connections 2024-05-09T08:27:57.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.092+0000 7fb2c6a00640 1 -- 172.21.15.133:0/3413617522 wait complete. 2024-05-09T08:27:57.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T08:27:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T08:27:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T08:27:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T08:27:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T08:27:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/649349846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:56 smithi160 ceph-mon[34070]: pgmap v315: 97 pgs: 17 creating+peering, 55 unknown, 25 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/649349846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:56 smithi188 ceph-mon[34645]: pgmap v315: 97 pgs: 17 creating+peering, 55 unknown, 25 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-05-09T08:27:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 -- 172.21.15.133:0/2465328935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 msgr2=0x7f4f48101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 --2- 172.21.15.133:0/2465328935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48101ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ca0 tx=0x7f4f3802a8c0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 -- 172.21.15.133:0/2465328935 shutdown_connections 2024-05-09T08:27:57.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 --2- 172.21.15.133:0/2465328935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48103430 0x7f4f48109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 --2- 172.21.15.133:0/2465328935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48102a70 0x7f4f48102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 --2- 172.21.15.133:0/2465328935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48101ca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 -- 172.21.15.133:0/2465328935 >> 172.21.15.133:0/2465328935 conn(0x7f4f480fce10 msgr2=0x7f4f480ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 -- 172.21.15.133:0/2465328935 shutdown_connections 2024-05-09T08:27:57.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.317+0000 7f4f4e600640 1 -- 172.21.15.133:0/2465328935 wait complete. 2024-05-09T08:27:57.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.318+0000 7f4f4e600640 1 Processor -- start 2024-05-09T08:27:57.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.318+0000 7f4f4e600640 1 -- start start 2024-05-09T08:27:57.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48102a70 0x7f4f4819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48103430 0x7f4f481a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4810d4d0 con 0x7f4f48101880 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4810d350 con 0x7f4f48102a70 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f4e600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4810d650 con 0x7f4f48103430 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f47400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f47400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33208/0 (socket says 172.21.15.133:33208) 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f47400640 1 -- 172.21.15.133:0/1532833454 learned_addr learned my addr 172.21.15.133:0/1532833454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f47e00640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48103430 0x7f4f481a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.319+0000 7f4f3ea00640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48102a70 0x7f4f4819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 -- 172.21.15.133:0/1532833454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48103430 msgr2=0x7f4f481a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48103430 0x7f4f481a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 -- 172.21.15.133:0/1532833454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48102a70 msgr2=0x7f4f4819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:57.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48102a70 0x7f4f4819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 -- 172.21.15.133:0/1532833454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f481a1a00 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f47400640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009bf0 tx=0x7f4f38031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3802fa20 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f3802fbc0 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f481a1c90 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38046750 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.320+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481a2180 con 0x7f4f48101880 2024-05-09T08:27:57.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.322+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f481a23b0 con 0x7f4f48101880 2024-05-09T08:27:57.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.323+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4f3802fd70 con 0x7f4f48101880 2024-05-09T08:27:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.325+0000 7f4f44c00640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 0x7f4f2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.325+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4f380f0210 con 0x7f4f48101880 2024-05-09T08:27:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.325+0000 7f4f3ea00640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 0x7f4f2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:57.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.326+0000 7f4f3ea00640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 0x7f4f2c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f4819b2f0 tx=0x7f4f30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:57.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.330+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f380bd220 con 0x7f4f48101880 2024-05-09T08:27:57.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:57.569+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4f481a2710 con 0x7f4f48101880 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:active 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3413617522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: Cluster is now healthy 2024-05-09T08:27:57.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:57 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:active 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3413617522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: Cluster is now healthy 2024-05-09T08:27:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:57 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T08:27:58.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] up:active 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3413617522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: Cluster is now healthy 2024-05-09T08:27:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:57 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T08:27:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.222+0000 7f4f44c00640 1 -- 172.21.15.133:0/1532833454 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f4f380bcb20 con 0x7f4f48101880 2024-05-09T08:27:58.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 msgr2=0x7f4f2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 0x7f4f2c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f4819b2f0 tx=0x7f4f30005f90 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 msgr2=0x7f4f48199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009bf0 tx=0x7f4f38031ee0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 shutdown_connections 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4f2c0a2c10 0x7f4f2c0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f48103430 0x7f4f481a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f48102a70 0x7f4f4819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 --2- 172.21.15.133:0/1532833454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f48101880 0x7f4f48199de0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.225+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 >> 172.21.15.133:0/1532833454 conn(0x7f4f480fce10 msgr2=0x7f4f48073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.226+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 shutdown_connections 2024-05-09T08:27:58.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.226+0000 7f4f4e600640 1 -- 172.21.15.133:0/1532833454 wait complete. 2024-05-09T08:27:58.248 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-09T08:27:58.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-09T08:27:58.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 -- 172.21.15.133:0/805064590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 msgr2=0x7f16d0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:58.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 --2- 172.21.15.133:0/805064590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d0109cb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009860 tx=0x7f16c802a340 comp rx=0 tx=0).stop 2024-05-09T08:27:58.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 -- 172.21.15.133:0/805064590 shutdown_connections 2024-05-09T08:27:58.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 --2- 172.21.15.133:0/805064590 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d0109cb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 --2- 172.21.15.133:0/805064590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0102a60 0x7f16d0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 --2- 172.21.15.133:0/805064590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0101870 0x7f16d0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 -- 172.21.15.133:0/805064590 >> 172.21.15.133:0/805064590 conn(0x7f16d00fcde0 msgr2=0x7f16d00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:58.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 -- 172.21.15.133:0/805064590 shutdown_connections 2024-05-09T08:27:58.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.449+0000 7f16d8400640 1 -- 172.21.15.133:0/805064590 wait complete. 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 Processor -- start 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 -- start start 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0101870 0x7f16d0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d4f0 con 0x7f16d0102a60 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d370 con 0x7f16d0103420 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d8400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010d670 con 0x7f16d0101870 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d5000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.450+0000 7f16d5000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33226/0 (socket says 172.21.15.133:33226) 2024-05-09T08:27:58.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 -- 172.21.15.133:0/764627125 learned_addr learned my addr 172.21.15.133:0/764627125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5a00640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0101870 0x7f16d0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d6400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 -- 172.21.15.133:0/764627125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0101870 msgr2=0x7f16d0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0101870 0x7f16d0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 -- 172.21.15.133:0/764627125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 msgr2=0x7f16d019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 -- 172.21.15.133:0/764627125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d019f790 con 0x7f16d0102a60 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16d5000640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e9f0 tx=0x7f16bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00cd80 con 0x7f16d0102a60 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc005500 con 0x7f16d0102a60 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.451+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc010690 con 0x7f16d0102a60 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.452+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d019fa80 con 0x7f16d0102a60 2024-05-09T08:27:58.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.452+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d019ffd0 con 0x7f16d0102a60 2024-05-09T08:27:58.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.453+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d0108bb0 con 0x7f16d0102a60 2024-05-09T08:27:58.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.455+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f16bc0046c0 con 0x7f16d0102a60 2024-05-09T08:27:58.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.457+0000 7f16ce000640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 0x7f16b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:58.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.457+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f16bc0cf270 con 0x7f16d0102a60 2024-05-09T08:27:58.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.457+0000 7f16d5a00640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 0x7f16b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:58.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.458+0000 7f16d5a00640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 0x7f16b40a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098d0 tx=0x7f16c0010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:58.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.461+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16bc09c320 con 0x7f16d0102a60 2024-05-09T08:27:58.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:58.701+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f16d002d3e0 con 0x7f16d0102a60 2024-05-09T08:27:59.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.234+0000 7f16ce000640 1 -- 172.21.15.133:0/764627125 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f16bc086070 con 0x7f16d0102a60 2024-05-09T08:27:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 msgr2=0x7f16b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 0x7f16b40a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f16c00098d0 tx=0x7f16c0010040 comp rx=0 tx=0).stop 2024-05-09T08:27:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 msgr2=0x7f16d01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e9f0 tx=0x7f16bc00edf0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 shutdown_connections 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f16b40a2c10 0x7f16b40a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16d0103420 0x7f16d019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16d0102a60 0x7f16d01980b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 --2- 172.21.15.133:0/764627125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d0101870 0x7f16d0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 >> 172.21.15.133:0/764627125 conn(0x7f16d00fcde0 msgr2=0x7f16d0105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.240+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 shutdown_connections 2024-05-09T08:27:59.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.241+0000 7f16d8400640 1 -- 172.21.15.133:0/764627125 wait complete. 2024-05-09T08:27:59.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-09T08:27:59.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 -- 172.21.15.133:0/148154482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390108f60 msgr2=0x7f43901093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 --2- 172.21.15.133:0/148154482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390108f60 0x7f43901093e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4380009900 tx=0x7f438002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 -- 172.21.15.133:0/148154482 shutdown_connections 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 --2- 172.21.15.133:0/148154482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390109920 0x7f43901101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 --2- 172.21.15.133:0/148154482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390108f60 0x7f43901093e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 --2- 172.21.15.133:0/148154482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390107d70 0x7f4390108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 -- 172.21.15.133:0/148154482 >> 172.21.15.133:0/148154482 conn(0x7f4390076d00 msgr2=0x7f4390079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 -- 172.21.15.133:0/148154482 shutdown_connections 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.454+0000 7f4397200640 1 -- 172.21.15.133:0/148154482 wait complete. 2024-05-09T08:27:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 Processor -- start 2024-05-09T08:27:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 -- start start 2024-05-09T08:27:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390108f60 0x7f439019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43901139f0 con 0x7f4390109920 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113870 con 0x7f4390108f60 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.455+0000 7f4397200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113b70 con 0x7f4390107d70 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4396800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:55374/0 (socket says 172.21.15.133:55374) 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 -- 172.21.15.133:0/608551904 learned_addr learned my addr 172.21.15.133:0/608551904 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4396800640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33244/0 (socket says 172.21.15.133:33244) 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395400640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390108f60 0x7f439019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 -- 172.21.15.133:0/608551904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390108f60 msgr2=0x7f439019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390108f60 0x7f439019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 -- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 msgr2=0x7f43901a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 -- 172.21.15.133:0/608551904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43901a5c30 con 0x7f4390107d70 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4396800640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.456+0000 7f4395e00640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f438400e9c0 tx=0x7f438400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.457+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438400cd80 con 0x7f4390107d70 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.457+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43840035f0 con 0x7f4390107d70 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.457+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43901a5f20 con 0x7f4390107d70 2024-05-09T08:27:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.457+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384010690 con 0x7f4390107d70 2024-05-09T08:27:59.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.457+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43901a6380 con 0x7f4390107d70 2024-05-09T08:27:59.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.458+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439010f090 con 0x7f4390107d70 2024-05-09T08:27:59.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.460+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f43840037a0 con 0x7f4390107d70 2024-05-09T08:27:59.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.461+0000 7f438aa00640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 0x7f43780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:27:59.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.461+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f43840cf400 con 0x7f4390107d70 2024-05-09T08:27:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.462+0000 7f4395400640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 0x7f43780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:27:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.462+0000 7f4395400640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 0x7f43780a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43800314e0 tx=0x7f4380000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:27:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.463+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f438409c4a0 con 0x7f4390107d70 2024-05-09T08:27:59.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:59 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T08:27:59.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:59 smithi160 ceph-mon[34070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:59.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:59 smithi160 ceph-mon[34070]: pgmap v317: 97 pgs: 17 creating+peering, 40 unknown, 40 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 625 B/s wr, 1 op/s 2024-05-09T08:27:59.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:27:59 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T08:27:59.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:59 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T08:27:59.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:59 smithi133 ceph-mon[29610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:59.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:59 smithi133 ceph-mon[29610]: pgmap v317: 97 pgs: 17 creating+peering, 40 unknown, 40 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 625 B/s wr, 1 op/s 2024-05-09T08:27:59.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:27:59 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T08:27:59.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:59 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1532833454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T08:27:59.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:59 smithi188 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:27:59.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:59 smithi188 ceph-mon[34645]: pgmap v317: 97 pgs: 17 creating+peering, 40 unknown, 40 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 625 B/s wr, 1 op/s 2024-05-09T08:27:59.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:27:59 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T08:27:59.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:27:59.696+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f439002d1c0 con 0x7f4390107d70 2024-05-09T08:28:00.163 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:27:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: 2024-05-09T08:27:59.719+0000 7f34240d1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T08:28:00.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.248+0000 7f438aa00640 1 -- 172.21.15.133:0/608551904 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f438409bda0 con 0x7f4390107d70 2024-05-09T08:28:00.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.253+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 msgr2=0x7f43780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.253+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 0x7f43780a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43800314e0 tx=0x7f4380000bc0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.253+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 msgr2=0x7f439019e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.253+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f438400e9c0 tx=0x7f438400ee80 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 shutdown_connections 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f43780a2b90 0x7f43780a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4390109920 0x7f43901a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4390108f60 0x7f439019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 --2- 172.21.15.133:0/608551904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4390107d70 0x7f439019e010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 >> 172.21.15.133:0/608551904 conn(0x7f4390076d00 msgr2=0x7f439010c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 shutdown_connections 2024-05-09T08:28:00.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.254+0000 7f4397200640 1 -- 172.21.15.133:0/608551904 wait complete. 2024-05-09T08:28:00.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-09T08:28:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 -- 172.21.15.133:0/2747709388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c103a90 msgr2=0x7f6f4c103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 --2- 172.21.15.133:0/2747709388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c103a90 0x7f6f4c103eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009900 tx=0x7f6f3c02a410 comp rx=0 tx=0).stop 2024-05-09T08:28:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 -- 172.21.15.133:0/2747709388 shutdown_connections 2024-05-09T08:28:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 --2- 172.21.15.133:0/2747709388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c105640 0x7f6f4c10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 --2- 172.21.15.133:0/2747709388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 --2- 172.21.15.133:0/2747709388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c103a90 0x7f6f4c103eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 -- 172.21.15.133:0/2747709388 >> 172.21.15.133:0/2747709388 conn(0x7f6f4c0ff020 msgr2=0x7f6f4c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 -- 172.21.15.133:0/2747709388 shutdown_connections 2024-05-09T08:28:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.457+0000 7f6f52400640 1 -- 172.21.15.133:0/2747709388 wait complete. 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 Processor -- start 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 -- start start 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 0x7f6f4c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c105640 0x7f6f4c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10f710 con 0x7f6f4c105640 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10f590 con 0x7f6f4c104c80 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.458+0000 7f6f52400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10f890 con 0x7f6f4c103a90 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c105640 0x7f6f4c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34132/0 (socket says 172.21.15.133:34132) 2024-05-09T08:28:00.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 -- 172.21.15.133:0/4004484991 learned_addr learned my addr 172.21.15.133:0/4004484991 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f4b400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 0x7f6f4c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 -- 172.21.15.133:0/4004484991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 msgr2=0x7f6f4c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 0x7f6f4c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 -- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c105640 msgr2=0x7f6f4c1a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c105640 0x7f6f4c1a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 -- 172.21.15.133:0/4004484991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c1a19d0 con 0x7f6f4c104c80 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f4b400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 0x7f6f4c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f43e00640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400e9f0 tx=0x7f6f3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400cd80 con 0x7f6f4c104c80 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34010910 con 0x7f6f4c104c80 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.459+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c1a1cc0 con 0x7f6f4c104c80 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.460+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34018990 con 0x7f6f4c104c80 2024-05-09T08:28:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.460+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a21c0 con 0x7f6f4c104c80 2024-05-09T08:28:00.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.461+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c10ae30 con 0x7f6f4c104c80 2024-05-09T08:28:00.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.462+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6f34010ac0 con 0x7f6f4c104c80 2024-05-09T08:28:00.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.464+0000 7f6f48c00640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 0x7f6f300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:00.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.464+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6f340cf8a0 con 0x7f6f4c104c80 2024-05-09T08:28:00.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.464+0000 7f6f4b400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 0x7f6f300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:00.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.465+0000 7f6f4b400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 0x7f6f300a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006010 tx=0x7f6f3c032cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:00.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.466+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f3409c8b0 con 0x7f6f4c104c80 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-09T08:28:00.609 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-09T08:28:00.610 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/608551904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.610 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:00 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/608551904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:00 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/764627125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/608551904' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:00 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T08:28:00.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:00.690+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6f4c02d1c0 con 0x7f6f4c104c80 2024-05-09T08:28:01.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.263+0000 7f6f48c00640 1 -- 172.21.15.133:0/4004484991 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f6f3409c1b0 con 0x7f6f4c104c80 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 msgr2=0x7f6f300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 0x7f6f300a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006010 tx=0x7f6f3c032cf0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 msgr2=0x7f6f4c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f3400e9f0 tx=0x7f6f3400eeb0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 shutdown_connections 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f6f300a2b90 0x7f6f300a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c105640 0x7f6f4c1a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c104c80 0x7f6f4c19a2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 --2- 172.21.15.133:0/4004484991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f4c103a90 0x7f6f4c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 >> 172.21.15.133:0/4004484991 conn(0x7f6f4c0ff020 msgr2=0x7f6f4c108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:01.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 shutdown_connections 2024-05-09T08:28:01.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.267+0000 7f6f52400640 1 -- 172.21.15.133:0/4004484991 wait complete. 2024-05-09T08:28:01.289 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-09T08:28:01.289 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-09T08:28:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 -- 172.21.15.133:0/1460182946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c10107e90 msgr2=0x7f1c101082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 --2- 172.21.15.133:0/1460182946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c10107e90 0x7f1c101082b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 -- 172.21.15.133:0/1460182946 shutdown_connections 2024-05-09T08:28:01.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 --2- 172.21.15.133:0/1460182946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c101099a0 0x7f1c101102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 --2- 172.21.15.133:0/1460182946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10108fe0 0x7f1c10109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 --2- 172.21.15.133:0/1460182946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c10107e90 0x7f1c101082b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 -- 172.21.15.133:0/1460182946 >> 172.21.15.133:0/1460182946 conn(0x7f1c10076e30 msgr2=0x7f1c10079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:01.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 -- 172.21.15.133:0/1460182946 shutdown_connections 2024-05-09T08:28:01.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.484+0000 7f1c17000640 1 -- 172.21.15.133:0/1460182946 wait complete. 2024-05-09T08:28:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.485+0000 7f1c17000640 1 Processor -- start 2024-05-09T08:28:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 -- start start 2024-05-09T08:28:01.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10107e90 0x7f1c1019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 0x7f1c1019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113b80 con 0x7f1c10108fe0 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113a00 con 0x7f1c101099a0 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c17000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10113d00 con 0x7f1c10107e90 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c0f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 0x7f1c1019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c15000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c0f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 0x7f1c1019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33274/0 (socket says 172.21.15.133:33274) 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c0f400640 1 -- 172.21.15.133:0/1712721929 learned_addr learned my addr 172.21.15.133:0/1712721929 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c15000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34160/0 (socket says 172.21.15.133:34160) 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.486+0000 7f1c0fe00640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10107e90 0x7f1c1019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 -- 172.21.15.133:0/1712721929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10107e90 msgr2=0x7f1c1019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10107e90 0x7f1c1019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 -- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 msgr2=0x7f1c1019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 0x7f1c1019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 -- 172.21.15.133:0/1712721929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c101a5de0 con 0x7f1c101099a0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c15000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009fd0 tx=0x7f1c0800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0800ce20 con 0x7f1c101099a0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c08005020 con 0x7f1c101099a0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08010720 con 0x7f1c101099a0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a60d0 con 0x7f1c101099a0 2024-05-09T08:28:01.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.487+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a65d0 con 0x7f1c101099a0 2024-05-09T08:28:01.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.488+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1010f1a0 con 0x7f1c101099a0 2024-05-09T08:28:01.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.490+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1c08005540 con 0x7f1c101099a0 2024-05-09T08:28:01.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.492+0000 7f1c0cc00640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 0x7f1bf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.492+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1c080cf610 con 0x7f1c101099a0 2024-05-09T08:28:01.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.492+0000 7f1c0fe00640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 0x7f1bf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.493+0000 7f1c0fe00640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 0x7f1bf40a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031440 tx=0x7f1c000315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:01.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.495+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c0809c620 con 0x7f1c101099a0 2024-05-09T08:28:01.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:active 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: Cluster is now healthy 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4004484991' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:01 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:active 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: Cluster is now healthy 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-09T08:28:01.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-09T08:28:01.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-09T08:28:01.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4004484991' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.679 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:01 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] up:active 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: Cluster is now healthy 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4004484991' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:01 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T08:28:01.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.723+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1c10061c70 con 0x7f1bf40a2c10 2024-05-09T08:28:01.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.753+0000 7f1c0cc00640 1 -- 172.21.15.133:0/1712721929 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c10061c70 con 0x7f1bf40a2c10 2024-05-09T08:28:01.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 msgr2=0x7f1bf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 0x7f1bf40a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031440 tx=0x7f1c000315b0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 msgr2=0x7f1c101a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009fd0 tx=0x7f1c0800ef10 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 shutdown_connections 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1bf40a2c10 0x7f1bf40a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c101099a0 0x7f1c101a5770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c10108fe0 0x7f1c1019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 --2- 172.21.15.133:0/1712721929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c10107e90 0x7f1c1019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 >> 172.21.15.133:0/1712721929 conn(0x7f1c10076e30 msgr2=0x7f1c1010c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 shutdown_connections 2024-05-09T08:28:01.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.756+0000 7f1c17000640 1 -- 172.21.15.133:0/1712721929 wait complete. 2024-05-09T08:28:01.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 -- 172.21.15.133:0/1058039610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 msgr2=0x7f089c1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 --2- 172.21.15.133:0/1058039610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c1093e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0884009900 tx=0x7f088402a410 comp rx=0 tx=0).stop 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 -- 172.21.15.133:0/1058039610 shutdown_connections 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 --2- 172.21.15.133:0/1058039610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 0x7f089c1101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 --2- 172.21.15.133:0/1058039610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c1093e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 --2- 172.21.15.133:0/1058039610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 -- 172.21.15.133:0/1058039610 >> 172.21.15.133:0/1058039610 conn(0x7f089c076d00 msgr2=0x7f089c079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 -- 172.21.15.133:0/1058039610 shutdown_connections 2024-05-09T08:28:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.971+0000 7f089be00640 1 -- 172.21.15.133:0/1058039610 wait complete. 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.972+0000 7f089be00640 1 Processor -- start 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.972+0000 7f089be00640 1 -- start start 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.972+0000 7f089be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.972+0000 7f089be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.972+0000 7f089be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 0x7f089c1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113a20 con 0x7f089c109920 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089be00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1138a0 con 0x7f089c108f60 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f089c113ba0 con 0x7f089c107d70 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 0x7f089c1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34198/0 (socket says 172.21.15.133:34198) 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 -- 172.21.15.133:0/1259044141 learned_addr learned my addr 172.21.15.133:0/1259044141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089aa00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 -- 172.21.15.133:0/1259044141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 msgr2=0x7f089c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 -- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 msgr2=0x7f089c1a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 0x7f089c1a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f0892000640 1 -- 172.21.15.133:0/1259044141 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c1a5c50 con 0x7f089c108f60 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.973+0000 7f089aa00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f0892000640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0884031440 tx=0x7f0884032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088403f030 con 0x7f089c108f60 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0884003ce0 con 0x7f089c108f60 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c1a5ee0 con 0x7f089c108f60 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884035070 con 0x7f089c108f60 2024-05-09T08:28:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.974+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a6350 con 0x7f089c108f60 2024-05-09T08:28:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.975+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c10f030 con 0x7f089c108f60 2024-05-09T08:28:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.977+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0884042030 con 0x7f089c108f60 2024-05-09T08:28:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.978+0000 7f08a0a00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 0x7f08800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.978+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f08840ef370 con 0x7f089c108f60 2024-05-09T08:28:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.978+0000 7f089aa00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 0x7f08800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:01.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.979+0000 7f089aa00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 0x7f08800a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f089c108dc0 tx=0x7f088c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:01.982+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08840bbd30 con 0x7f089c108f60 2024-05-09T08:28:02.178 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:28:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:28:01.732+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:28:02.178 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:28:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:28:01.732+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:28:02.178 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:28:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:28:01.732+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:28:02.178 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:28:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:28:01.732+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:28:02.178 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:28:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:28:01.732+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:28:02.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.211+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f089c061d00 con 0x7f08800a2c10 2024-05-09T08:28:02.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.252+0000 7f08a0a00640 1 -- 172.21.15.133:0/1259044141 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f089c061d00 con 0x7f08800a2c10 2024-05-09T08:28:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 msgr2=0x7f08800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 0x7f08800a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f089c108dc0 tx=0x7f088c005f90 comp rx=0 tx=0).stop 2024-05-09T08:28:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 msgr2=0x7f089c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0884031440 tx=0x7f0884032ce0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 shutdown_connections 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08800a2c10 0x7f08800a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f089c109920 0x7f089c1a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f089c108f60 0x7f089c19e570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 --2- 172.21.15.133:0/1259044141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f089c107d70 0x7f089c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 >> 172.21.15.133:0/1259044141 conn(0x7f089c076d00 msgr2=0x7f089c10c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 shutdown_connections 2024-05-09T08:28:02.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.255+0000 7f089be00640 1 -- 172.21.15.133:0/1259044141 wait complete. 2024-05-09T08:28:02.276 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.482+0000 7fc68a800640 1 -- 172.21.15.133:0/1392365802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc684100640 msgr2=0x7fc684100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.482+0000 7fc68a800640 1 --2- 172.21.15.133:0/1392365802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc684100640 0x7fc684100a60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009be0 tx=0x7fc67c02a6c0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 -- 172.21.15.133:0/1392365802 shutdown_connections 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 --2- 172.21.15.133:0/1392365802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6841021f0 0x7fc684108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 --2- 172.21.15.133:0/1392365802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684101830 0x7fc684101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 --2- 172.21.15.133:0/1392365802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc684100640 0x7fc684100a60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 -- 172.21.15.133:0/1392365802 >> 172.21.15.133:0/1392365802 conn(0x7fc6840fbbd0 msgr2=0x7fc6840fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 -- 172.21.15.133:0/1392365802 shutdown_connections 2024-05-09T08:28:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.483+0000 7fc68a800640 1 -- 172.21.15.133:0/1392365802 wait complete. 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.484+0000 7fc68a800640 1 Processor -- start 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.484+0000 7fc68a800640 1 -- start start 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.484+0000 7fc68a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.484+0000 7fc68a800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 0x7fc684198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.484+0000 7fc68a800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6841021f0 0x7fc68419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc68a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc68410c2c0 con 0x7fc684100640 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc68a800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc68410c140 con 0x7fc6841021f0 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc68a800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc68410c440 con 0x7fc684101830 2024-05-09T08:28:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33330/0 (socket says 172.21.15.133:33330) 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 -- 172.21.15.133:0/4031906180 learned_addr learned my addr 172.21.15.133:0/4031906180 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc688a00640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 0x7fc684198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689e00640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6841021f0 0x7fc68419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 -- 172.21.15.133:0/4031906180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 msgr2=0x7fc684198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 0x7fc684198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 -- 172.21.15.133:0/4031906180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6841021f0 msgr2=0x7fc68419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6841021f0 0x7fc68419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc689400640 1 -- 172.21.15.133:0/4031906180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68419f690 con 0x7fc684100640 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.485+0000 7fc688a00640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 0x7fc684198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc689400640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc67c02acb0 tx=0x7fc67c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c041020 con 0x7fc684100640 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68419f920 con 0x7fc684100640 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c031920 con 0x7fc684100640 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68419fe60 con 0x7fc684100640 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.486+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c036040 con 0x7fc684100640 2024-05-09T08:28:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.487+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6841a0030 con 0x7fc684100640 2024-05-09T08:28:02.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.490+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc67c044020 con 0x7fc684100640 2024-05-09T08:28:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.491+0000 7fc682000640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 0x7fc6680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.491+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc67c0effd0 con 0x7fc684100640 2024-05-09T08:28:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.491+0000 7fc688a00640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 0x7fc6680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.492+0000 7fc688a00640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 0x7fc6680a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6700096c0 tx=0x7fc670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:02.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.495+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc67c04b020 con 0x7fc684100640 2024-05-09T08:28:02.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:02 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T08:28:02.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:02 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:active 2024-05-09T08:28:02.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:02 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-09T08:28:02.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:02 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:28:02.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:02 smithi160 ceph-mon[34070]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:02.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:02 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T08:28:02.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:02 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:active 2024-05-09T08:28:02.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:02 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-09T08:28:02.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:02 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:28:02.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:02 smithi133 ceph-mon[29610]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:02.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:02 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T08:28:02.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:02 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] up:active 2024-05-09T08:28:02.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:02 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-05-09T08:28:02.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:02 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:28:02.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:02 smithi188 ceph-mon[34645]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:02.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.725+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc6840fdd90 con 0x7fc6680a2b90 2024-05-09T08:28:02.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.734+0000 7fc682000640 1 -- 172.21.15.133:0/4031906180 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc6840fdd90 con 0x7fc6680a2b90 2024-05-09T08:28:02.734 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/qa/sv_0/9612a2b3-61db-4fb8-9a69-b455724f2556 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 msgr2=0x7fc6680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 0x7fc6680a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6700096c0 tx=0x7fc670009260 comp rx=0 tx=0).stop 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 msgr2=0x7fc684197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc67c02acb0 tx=0x7fc67c0305d0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 shutdown_connections 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc6680a2b90 0x7fc6680a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6841021f0 0x7fc68419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc684101830 0x7fc684198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 --2- 172.21.15.133:0/4031906180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc684100640 0x7fc684197ae0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 >> 172.21.15.133:0/4031906180 conn(0x7fc6840fbbd0 msgr2=0x7fc6840fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 shutdown_connections 2024-05-09T08:28:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.737+0000 7fc68a800640 1 -- 172.21.15.133:0/4031906180 wait complete. 2024-05-09T08:28:02.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T08:28:02.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 -- 172.21.15.133:0/4212934906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 msgr2=0x7fe164103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 --2- 172.21.15.133:0/4212934906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164103eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402a3f0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 -- 172.21.15.133:0/4212934906 shutdown_connections 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 --2- 172.21.15.133:0/4212934906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe16410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 --2- 172.21.15.133:0/4212934906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe164105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 --2- 172.21.15.133:0/4212934906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164103eb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 -- 172.21.15.133:0/4212934906 >> 172.21.15.133:0/4212934906 conn(0x7fe1640ff020 msgr2=0x7fe164101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 -- 172.21.15.133:0/4212934906 shutdown_connections 2024-05-09T08:28:02.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.965+0000 7fe16a600640 1 -- 172.21.15.133:0/4212934906 wait complete. 2024-05-09T08:28:02.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.966+0000 7fe16a600640 1 Processor -- start 2024-05-09T08:28:02.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 -- start start 2024-05-09T08:28:02.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe16419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe1641a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410f6e0 con 0x7fe164103a90 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410f560 con 0x7fe164105640 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe16a600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe16410f860 con 0x7fe164104c80 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.967+0000 7fe163400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33350/0 (socket says 172.21.15.133:33350) 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 -- 172.21.15.133:0/2029687449 learned_addr learned my addr 172.21.15.133:0/2029687449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe15be00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe16419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163e00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe1641a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 -- 172.21.15.133:0/2029687449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 msgr2=0x7fe16419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe16419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 -- 172.21.15.133:0/2029687449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 msgr2=0x7fe1641a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:02.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe1641a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:02.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 -- 172.21.15.133:0/2029687449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1641a19e0 con 0x7fe164103a90 2024-05-09T08:28:02.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe15be00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe16419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:02.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163e00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe1641a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:02.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.968+0000 7fe163400640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe154031610 tx=0x7fe154004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.969+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15403f030 con 0x7fe164103a90 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.969+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154003ad0 con 0x7fe164103a90 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.969+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641a1c70 con 0x7fe164103a90 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.969+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154035070 con 0x7fe164103a90 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.969+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641a21b0 con 0x7fe164103a90 2024-05-09T08:28:02.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.970+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16410ae20 con 0x7fe164103a90 2024-05-09T08:28:02.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.973+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe154042030 con 0x7fe164103a90 2024-05-09T08:28:02.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.974+0000 7fe160c00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 0x7fe1480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:02.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.975+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe1540ef4b0 con 0x7fe164103a90 2024-05-09T08:28:02.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.975+0000 7fe15be00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 0x7fe1480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:02.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.976+0000 7fe15be00640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 0x7fe1480a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0095f0 tx=0x7fe14c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:02.978+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1540bc4c0 con 0x7fe164103a90 2024-05-09T08:28:03.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.208+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe164061d00 con 0x7fe1480a2c10 2024-05-09T08:28:03.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.251+0000 7fe160c00640 1 -- 172.21.15.133:0/2029687449 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe164061d00 con 0x7fe1480a2c10 2024-05-09T08:28:03.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 msgr2=0x7fe1480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 0x7fe1480a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0095f0 tx=0x7fe14c005f90 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 msgr2=0x7fe164199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe154031610 tx=0x7fe154004270 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 shutdown_connections 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe1480a2c10 0x7fe1480a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe164105640 0x7fe1641a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe164104c80 0x7fe16419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 --2- 172.21.15.133:0/2029687449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe164103a90 0x7fe164199dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 >> 172.21.15.133:0/2029687449 conn(0x7fe1640ff020 msgr2=0x7fe164107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 shutdown_connections 2024-05-09T08:28:03.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.254+0000 7fe16a600640 1 -- 172.21.15.133:0/2029687449 wait complete. 2024-05-09T08:28:03.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T08:28:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 -- 172.21.15.133:0/1372137149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 msgr2=0x7fa28c076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 --2- 172.21.15.133:0/1372137149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c076610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009860 tx=0x7fa27c02a340 comp rx=0 tx=0).stop 2024-05-09T08:28:03.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 -- 172.21.15.133:0/1372137149 shutdown_connections 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 --2- 172.21.15.133:0/1372137149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077dd0 0x7fa28c078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 --2- 172.21.15.133:0/1372137149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077430 0x7fa28c077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 --2- 172.21.15.133:0/1372137149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c076610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 -- 172.21.15.133:0/1372137149 >> 172.21.15.133:0/1372137149 conn(0x7fa28c0fefe0 msgr2=0x7fa28c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 -- 172.21.15.133:0/1372137149 shutdown_connections 2024-05-09T08:28:03.477 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.476+0000 7fa293a00640 1 -- 172.21.15.133:0/1372137149 wait complete. 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.477+0000 7fa293a00640 1 Processor -- start 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.477+0000 7fa293a00640 1 -- start start 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c19e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077dd0 0x7fa28c1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c113b20 con 0x7fa28c077dd0 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c1139a0 con 0x7fa28c077430 2024-05-09T08:28:03.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa293a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c113ca0 con 0x7fa28c0761f0 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa28be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa291000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c19e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa291a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077dd0 0x7fa28c1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa28be00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:34252/0 (socket says 172.21.15.133:34252) 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa291000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c19e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:55488/0 (socket says 172.21.15.133:55488) 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.478+0000 7fa28be00640 1 -- 172.21.15.133:0/69336636 learned_addr learned my addr 172.21.15.133:0/69336636 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 -- 172.21.15.133:0/69336636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 msgr2=0x7fa28c19e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c19e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 -- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077dd0 msgr2=0x7fa28c1a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077dd0 0x7fa28c1a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 -- 172.21.15.133:0/69336636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c1a5d90 con 0x7fa28c077430 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa28be00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa27800e9c0 tx=0x7fa27800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27800ccc0 con 0x7fa28c077430 2024-05-09T08:28:03.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a6080 con 0x7fa28c077430 2024-05-09T08:28:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa278005500 con 0x7fa28c077430 2024-05-09T08:28:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278010430 con 0x7fa28c077430 2024-05-09T08:28:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.479+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a6620 con 0x7fa28c077430 2024-05-09T08:28:03.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.481+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c1105f0 con 0x7fa28c077430 2024-05-09T08:28:03.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.482+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa2780046c0 con 0x7fa28c077430 2024-05-09T08:28:03.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.483+0000 7fa289600640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 0x7fa2700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.484+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa2780cf2e0 con 0x7fa28c077430 2024-05-09T08:28:03.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.484+0000 7fa291000640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 0x7fa2700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.484+0000 7fa291000640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 0x7fa2700a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa27c02a930 tx=0x7fa27c030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:03.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.488+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa27809c190 con 0x7fa28c077430 2024-05-09T08:28:03.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:03 smithi160 ceph-mon[34070]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:03 smithi160 ceph-mon[34070]: pgmap v319: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s rd, 3.9 KiB/s wr, 12 op/s 2024-05-09T08:28:03.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:03 smithi160 ceph-mon[34070]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:03 smithi133 ceph-mon[29610]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:03 smithi133 ceph-mon[29610]: pgmap v319: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s rd, 3.9 KiB/s wr, 12 op/s 2024-05-09T08:28:03.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:03 smithi133 ceph-mon[29610]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:03 smithi188 ceph-mon[34645]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:03 smithi188 ceph-mon[34645]: pgmap v319: 97 pgs: 97 active+clean; 580 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s rd, 3.9 KiB/s wr, 12 op/s 2024-05-09T08:28:03.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:03 smithi188 ceph-mon[34645]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:03.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.721+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa28c061c70 con 0x7fa2700a2ca0 2024-05-09T08:28:03.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.731+0000 7fa289600640 1 -- 172.21.15.133:0/69336636 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa28c061c70 con 0x7fa2700a2ca0 2024-05-09T08:28:03.731 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/qa/sv_1/9eeaffab-bdfb-4244-a54d-91ba89170d72 2024-05-09T08:28:03.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 msgr2=0x7fa2700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 0x7fa2700a5160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa27c02a930 tx=0x7fa27c030570 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 msgr2=0x7fa28c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa27800e9c0 tx=0x7fa27800ee80 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 shutdown_connections 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa2700a2ca0 0x7fa2700a5160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa28c077dd0 0x7fa28c1a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa28c077430 0x7fa28c19e6b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 --2- 172.21.15.133:0/69336636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa28c0761f0 0x7fa28c19e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 >> 172.21.15.133:0/69336636 conn(0x7fa28c0fefe0 msgr2=0x7fa28c100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.734+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 shutdown_connections 2024-05-09T08:28:03.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.735+0000 7fa293a00640 1 -- 172.21.15.133:0/69336636 wait complete. 2024-05-09T08:28:03.754 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:28:03.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 -- 172.21.15.133:0/1961164021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 msgr2=0x7efe541102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 --2- 172.21.15.133:0/1961164021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541102c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009ca0 tx=0x7efe4c02a8b0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 -- 172.21.15.133:0/1961164021 shutdown_connections 2024-05-09T08:28:03.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 --2- 172.21.15.133:0/1961164021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541102c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 --2- 172.21.15.133:0/1961164021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54108fe0 0x7efe54109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 --2- 172.21.15.133:0/1961164021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54107e90 0x7efe541082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 -- 172.21.15.133:0/1961164021 >> 172.21.15.133:0/1961164021 conn(0x7efe54076e50 msgr2=0x7efe54079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:03.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 -- 172.21.15.133:0/1961164021 shutdown_connections 2024-05-09T08:28:03.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.969+0000 7efe5b800640 1 -- 172.21.15.133:0/1961164021 wait complete. 2024-05-09T08:28:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.970+0000 7efe5b800640 1 Processor -- start 2024-05-09T08:28:03.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.970+0000 7efe5b800640 1 -- start start 2024-05-09T08:28:03.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54108fe0 0x7efe5419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe54113b40 con 0x7efe541099a0 2024-05-09T08:28:03.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efe541139c0 con 0x7efe54107e90 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe5b800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe54113cc0 con 0x7efe54108fe0 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe59800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:35926/0 (socket says 172.21.15.133:35926) 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 -- 172.21.15.133:0/926381452 learned_addr learned my addr 172.21.15.133:0/926381452 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:03.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe4be00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54108fe0 0x7efe5419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 -- 172.21.15.133:0/926381452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54108fe0 msgr2=0x7efe5419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54108fe0 0x7efe5419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 -- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 msgr2=0x7efe541a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:03.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:03.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.971+0000 7efe58e00640 1 -- 172.21.15.133:0/926381452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a5d70 con 0x7efe54107e90 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe58e00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e9f0 tx=0x7efe4400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4400cd80 con 0x7efe54107e90 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a6060 con 0x7efe54107e90 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe44005500 con 0x7efe54107e90 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44010690 con 0x7efe54107e90 2024-05-09T08:28:03.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.972+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a65b0 con 0x7efe54107e90 2024-05-09T08:28:03.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.973+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5410f140 con 0x7efe54107e90 2024-05-09T08:28:03.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.975+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efe440046c0 con 0x7efe54107e90 2024-05-09T08:28:03.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.977+0000 7efe52000640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 0x7efe380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:03.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.977+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efe440cf3b0 con 0x7efe54107e90 2024-05-09T08:28:03.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.977+0000 7efe4be00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 0x7efe380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:03.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.977+0000 7efe4be00640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 0x7efe380a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe5419f660 tx=0x7efe3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:03.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:03.979+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe4409c260 con 0x7efe54107e90 2024-05-09T08:28:04.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.219+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5402d1c0 con 0x7efe54107e90 2024-05-09T08:28:04.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.221+0000 7efe52000640 1 -- 172.21.15.133:0/926381452 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7efe4409bb60 con 0x7efe54107e90 2024-05-09T08:28:04.221 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:04.221 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:28:04.221 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:28:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 msgr2=0x7efe380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 0x7efe380a5160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efe5419f660 tx=0x7efe3c005f90 comp rx=0 tx=0).stop 2024-05-09T08:28:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 msgr2=0x7efe5419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe4400e9f0 tx=0x7efe4400edf0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 shutdown_connections 2024-05-09T08:28:04.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efe380a2ca0 0x7efe380a5160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe541099a0 0x7efe541a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe54108fe0 0x7efe5419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 --2- 172.21.15.133:0/926381452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe54107e90 0x7efe5419e150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 >> 172.21.15.133:0/926381452 conn(0x7efe54076e50 msgr2=0x7efe5410c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 shutdown_connections 2024-05-09T08:28:04.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.224+0000 7efe5b800640 1 -- 172.21.15.133:0/926381452 wait complete. 2024-05-09T08:28:04.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:28:04.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 -- 172.21.15.133:0/2991100594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83580761f0 msgr2=0x7f8358076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 --2- 172.21.15.133:0/2991100594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83580761f0 0x7f8358076610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8348009900 tx=0x7f834802a410 comp rx=0 tx=0).stop 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 -- 172.21.15.133:0/2991100594 shutdown_connections 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 --2- 172.21.15.133:0/2991100594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8358077dd0 0x7f8358078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 --2- 172.21.15.133:0/2991100594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 0x7f8358077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 --2- 172.21.15.133:0/2991100594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83580761f0 0x7f8358076610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 -- 172.21.15.133:0/2991100594 >> 172.21.15.133:0/2991100594 conn(0x7f83580ff020 msgr2=0x7f8358101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 -- 172.21.15.133:0/2991100594 shutdown_connections 2024-05-09T08:28:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.433+0000 7f835de00640 1 -- 172.21.15.133:0/2991100594 wait complete. 2024-05-09T08:28:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.434+0000 7f835de00640 1 Processor -- start 2024-05-09T08:28:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.434+0000 7f835de00640 1 -- start start 2024-05-09T08:28:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.434+0000 7f835de00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.434+0000 7f835de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 0x7f835819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f835de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f835de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8358113b20 con 0x7f8358077dd0 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f835de00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f83581139a0 con 0x7f8358077430 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f835de00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8358113ca0 con 0x7f83580761f0 2024-05-09T08:28:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8357400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8357400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59534/0 (socket says 172.21.15.133:59534) 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:37884/0 (socket says 172.21.15.133:37884) 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8357400640 1 -- 172.21.15.133:0/3678737078 learned_addr learned my addr 172.21.15.133:0/3678737078 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356000640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 0x7f835819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 -- 172.21.15.133:0/3678737078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 msgr2=0x7f835819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 0x7f835819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 -- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 msgr2=0x7f83581a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8356a00640 1 -- 172.21.15.133:0/3678737078 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83581a5da0 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.435+0000 7f8357400640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f8356a00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8348031440 tx=0x7f8348004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834803f070 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581a6030 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8348003cd0 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348035070 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.436+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a6540 con 0x7f83580761f0 2024-05-09T08:28:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.437+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358110400 con 0x7f83580761f0 2024-05-09T08:28:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.439+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f834803b4f0 con 0x7f83580761f0 2024-05-09T08:28:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.441+0000 7f835ca00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 0x7f833c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.441+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f83480efa40 con 0x7f83580761f0 2024-05-09T08:28:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.441+0000 7f8356000640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 0x7f833c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.441+0000 7f8356000640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 0x7f833c0a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f835819f690 tx=0x7f8344005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:04.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.443+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83480bc2a0 con 0x7f83580761f0 2024-05-09T08:28:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.645+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f835802d390 con 0x7f83580761f0 2024-05-09T08:28:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.647+0000 7f835ca00640 1 -- 172.21.15.133:0/3678737078 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22287 (secure 0 0 0) 0x7f83480c05a0 con 0x7f83580761f0 2024-05-09T08:28:04.648 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:04.648 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":86,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:56.775532+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.650+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 msgr2=0x7f833c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.650+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 0x7f833c0a50e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f835819f690 tx=0x7f8344005f90 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.650+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 msgr2=0x7f835819e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.650+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8348031440 tx=0x7f8348004270 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 shutdown_connections 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f833c0a2c20 0x7f833c0a50e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8358077dd0 0x7f83581a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8358077430 0x7f835819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 --2- 172.21.15.133:0/3678737078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83580761f0 0x7f835819e180 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8348031440 tx=0x7f8348004270 comp rx=0 tx=0).stop 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 >> 172.21.15.133:0/3678737078 conn(0x7f83580ff020 msgr2=0x7f8358102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 shutdown_connections 2024-05-09T08:28:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.651+0000 7f835de00640 1 -- 172.21.15.133:0/3678737078 wait complete. 2024-05-09T08:28:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:04 smithi160 ceph-mon[34070]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:04 smithi160 ceph-mon[34070]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T08:28:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:04 smithi160 ceph-mon[34070]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:04 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/926381452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:28:04.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:04 smithi133 ceph-mon[29610]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:04 smithi133 ceph-mon[29610]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T08:28:04.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:04 smithi133 ceph-mon[29610]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.667 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:04 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/926381452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:28:04.670 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T08:28:04.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:28:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:04 smithi188 ceph-mon[34645]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:04 smithi188 ceph-mon[34645]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-09T08:28:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:04 smithi188 ceph-mon[34645]: from='client.24680 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:28:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:04 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/926381452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 -- 172.21.15.133:0/2327511306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 msgr2=0x7fb63c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 --2- 172.21.15.133:0/2327511306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c102d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009900 tx=0x7fb62c02a410 comp rx=0 tx=0).stop 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 -- 172.21.15.133:0/2327511306 shutdown_connections 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 --2- 172.21.15.133:0/2327511306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 --2- 172.21.15.133:0/2327511306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c102d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 --2- 172.21.15.133:0/2327511306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 0x7fb63c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 -- 172.21.15.133:0/2327511306 >> 172.21.15.133:0/2327511306 conn(0x7fb63c0fcc10 msgr2=0x7fb63c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.867+0000 7fb643000640 1 -- 172.21.15.133:0/2327511306 shutdown_connections 2024-05-09T08:28:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.868+0000 7fb643000640 1 -- 172.21.15.133:0/2327511306 wait complete. 2024-05-09T08:28:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.868+0000 7fb643000640 1 Processor -- start 2024-05-09T08:28:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.868+0000 7fb643000640 1 -- start start 2024-05-09T08:28:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 0x7fb63c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10d320 con 0x7fb63c102890 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10d1a0 con 0x7fb63c103250 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb643000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c10d4a0 con 0x7fb63c1016a0 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb641200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb641200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59560/0 (socket says 172.21.15.133:59560) 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb641200640 1 -- 172.21.15.133:0/3804271433 learned_addr learned my addr 172.21.15.133:0/3804271433 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb642600640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.869+0000 7fb641c00640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 0x7fb63c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 -- 172.21.15.133:0/3804271433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 msgr2=0x7fb63c0755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 0x7fb63c0755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 -- 172.21.15.133:0/3804271433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 msgr2=0x7fb63c074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 -- 172.21.15.133:0/3804271433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c0749f0 con 0x7fb63c102890 2024-05-09T08:28:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb642600640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb641200640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009230 tx=0x7fb62c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c03f030 con 0x7fb63c102890 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c003ce0 con 0x7fb63c102890 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63c1a1ae0 con 0x7fb63c102890 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c035070 con 0x7fb63c102890 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.870+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb63c1a2080 con 0x7fb63c102890 2024-05-09T08:28:04.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.872+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63c1089e0 con 0x7fb63c102890 2024-05-09T08:28:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.873+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb62c042030 con 0x7fb63c102890 2024-05-09T08:28:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.875+0000 7fb63a000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 0x7fb6240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.875+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb62c0ef4f0 con 0x7fb63c102890 2024-05-09T08:28:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.875+0000 7fb641c00640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 0x7fb6240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:04.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.876+0000 7fb641c00640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 0x7fb6240a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb63c1026f0 tx=0x7fb630009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:04.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:04.879+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb62c0bbd50 con 0x7fb63c102890 2024-05-09T08:28:05.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.065+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb63c061c70 con 0x7fb6240a2ca0 2024-05-09T08:28:05.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.082+0000 7fb63a000640 1 -- 172.21.15.133:0/3804271433 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258948 (secure 0 0 0) 0x7fb63c061c70 con 0x7fb6240a2ca0 2024-05-09T08:28:05.086 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:05.088 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-05-09T08:28:05.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 msgr2=0x7fb6240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 0x7fb6240a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb63c1026f0 tx=0x7fb630009500 comp rx=0 tx=0).stop 2024-05-09T08:28:05.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 msgr2=0x7fb63c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009230 tx=0x7fb62c0042f0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 shutdown_connections 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb6240a2ca0 0x7fb6240a5160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63c103250 0x7fb63c074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb63c102890 0x7fb63c073c20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 --2- 172.21.15.133:0/3804271433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb63c1016a0 0x7fb63c0755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 >> 172.21.15.133:0/3804271433 conn(0x7fb63c0fcc10 msgr2=0x7fb63c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 shutdown_connections 2024-05-09T08:28:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.087+0000 7fb643000640 1 -- 172.21.15.133:0/3804271433 wait complete. 2024-05-09T08:28:05.106 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-05-09T08:28:04.661321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596098,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":152,"num_read_kb":128,"num_write":313,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372828,"kb_used_data":11196,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568996,"statfs":{"total":1151940427776,"available":1151558651904,"internally_reserved":0,"allocated":11464704,"data_stored":7093715,"data_compressed":107103,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5730,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":132,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.882742"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906996+0000","last_change":"2024-05-09T08:27:55.828188+0000","last_active":"2024-05-09T08:27:57.906996+0000","last_peered":"2024-05-09T08:27:57.906996+0000","last_clean":"2024-05-09T08:27:57.906996+0000","last_became_active":"2024-05-09T08:27:55.827858+0000","last_became_peered":"2024-05-09T08:27:55.827858+0000","last_unstale":"2024-05-09T08:27:57.906996+0000","last_undegraded":"2024-05-09T08:27:57.906996+0000","last_fullsized":"2024-05-09T08:27:57.906996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:19.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912235+0000","last_change":"2024-05-09T08:27:55.821504+0000","last_active":"2024-05-09T08:27:57.912235+0000","last_peered":"2024-05-09T08:27:57.912235+0000","last_clean":"2024-05-09T08:27:57.912235+0000","last_became_active":"2024-05-09T08:27:55.818102+0000","last_became_peered":"2024-05-09T08:27:55.818102+0000","last_unstale":"2024-05-09T08:27:57.912235+0000","last_undegraded":"2024-05-09T08:27:57.912235+0000","last_fullsized":"2024-05-09T08:27:57.912235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:00.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911803+0000","last_change":"2024-05-09T08:27:55.817240+0000","last_active":"2024-05-09T08:27:57.911803+0000","last_peered":"2024-05-09T08:27:57.911803+0000","last_clean":"2024-05-09T08:27:57.911803+0000","last_became_active":"2024-05-09T08:27:55.816865+0000","last_became_peered":"2024-05-09T08:27:55.816865+0000","last_unstale":"2024-05-09T08:27:57.911803+0000","last_undegraded":"2024-05-09T08:27:57.911803+0000","last_fullsized":"2024-05-09T08:27:57.911803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:04.149531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.781033+0000","last_change":"2024-05-09T08:27:55.805704+0000","last_active":"2024-05-09T08:27:56.781033+0000","last_peered":"2024-05-09T08:27:56.781033+0000","last_clean":"2024-05-09T08:27:56.781033+0000","last_became_active":"2024-05-09T08:27:55.805344+0000","last_became_peered":"2024-05-09T08:27:55.805344+0000","last_unstale":"2024-05-09T08:27:56.781033+0000","last_undegraded":"2024-05-09T08:27:56.781033+0000","last_fullsized":"2024-05-09T08:27:56.781033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:02.418060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224767+0000","last_change":"2024-05-09T08:27:55.821747+0000","last_active":"2024-05-09T08:27:57.224767+0000","last_peered":"2024-05-09T08:27:57.224767+0000","last_clean":"2024-05-09T08:27:57.224767+0000","last_became_active":"2024-05-09T08:27:55.821378+0000","last_became_peered":"2024-05-09T08:27:55.821378+0000","last_unstale":"2024-05-09T08:27:57.224767+0000","last_undegraded":"2024-05-09T08:27:57.224767+0000","last_fullsized":"2024-05-09T08:27:57.224767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:53.470723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978180+0000","last_change":"2024-05-09T08:27:55.812827+0000","last_active":"2024-05-09T08:27:56.978180+0000","last_peered":"2024-05-09T08:27:56.978180+0000","last_clean":"2024-05-09T08:27:56.978180+0000","last_became_active":"2024-05-09T08:27:55.812412+0000","last_became_peered":"2024-05-09T08:27:55.812412+0000","last_unstale":"2024-05-09T08:27:56.978180+0000","last_undegraded":"2024-05-09T08:27:56.978180+0000","last_fullsized":"2024-05-09T08:27:56.978180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:20.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387427+0000","last_change":"2024-05-09T08:27:55.811536+0000","last_active":"2024-05-09T08:27:57.387427+0000","last_peered":"2024-05-09T08:27:57.387427+0000","last_clean":"2024-05-09T08:27:57.387427+0000","last_became_active":"2024-05-09T08:27:55.811181+0000","last_became_peered":"2024-05-09T08:27:55.811181+0000","last_unstale":"2024-05-09T08:27:57.387427+0000","last_undegraded":"2024-05-09T08:27:57.387427+0000","last_fullsized":"2024-05-09T08:27:57.387427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227101+0000","last_change":"2024-05-09T08:27:55.823372+0000","last_active":"2024-05-09T08:27:57.227101+0000","last_peered":"2024-05-09T08:27:57.227101+0000","last_clean":"2024-05-09T08:27:57.227101+0000","last_became_active":"2024-05-09T08:27:55.823052+0000","last_became_peered":"2024-05-09T08:27:55.823052+0000","last_unstale":"2024-05-09T08:27:57.227101+0000","last_undegraded":"2024-05-09T08:27:57.227101+0000","last_fullsized":"2024-05-09T08:27:57.227101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:18.677587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906989+0000","last_change":"2024-05-09T08:27:55.802154+0000","last_active":"2024-05-09T08:27:57.906989+0000","last_peered":"2024-05-09T08:27:57.906989+0000","last_clean":"2024-05-09T08:27:57.906989+0000","last_became_active":"2024-05-09T08:27:55.801501+0000","last_became_peered":"2024-05-09T08:27:55.801501+0000","last_unstale":"2024-05-09T08:27:57.906989+0000","last_undegraded":"2024-05-09T08:27:57.906989+0000","last_fullsized":"2024-05-09T08:27:57.906989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:42.009521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978464+0000","last_change":"2024-05-09T08:27:55.810294+0000","last_active":"2024-05-09T08:27:56.978464+0000","last_peered":"2024-05-09T08:27:56.978464+0000","last_clean":"2024-05-09T08:27:56.978464+0000","last_became_active":"2024-05-09T08:27:55.809825+0000","last_became_peered":"2024-05-09T08:27:55.809825+0000","last_unstale":"2024-05-09T08:27:56.978464+0000","last_undegraded":"2024-05-09T08:27:56.978464+0000","last_fullsized":"2024-05-09T08:27:56.978464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:39:39.846941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387653+0000","last_change":"2024-05-09T08:27:55.806122+0000","last_active":"2024-05-09T08:27:57.387653+0000","last_peered":"2024-05-09T08:27:57.387653+0000","last_clean":"2024-05-09T08:27:57.387653+0000","last_became_active":"2024-05-09T08:27:55.803844+0000","last_became_peered":"2024-05-09T08:27:55.803844+0000","last_unstale":"2024-05-09T08:27:57.387653+0000","last_undegraded":"2024-05-09T08:27:57.387653+0000","last_fullsized":"2024-05-09T08:27:57.387653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:40.692633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780958+0000","last_change":"2024-05-09T08:27:55.810184+0000","last_active":"2024-05-09T08:27:56.780958+0000","last_peered":"2024-05-09T08:27:56.780958+0000","last_clean":"2024-05-09T08:27:56.780958+0000","last_became_active":"2024-05-09T08:27:55.809833+0000","last_became_peered":"2024-05-09T08:27:55.809833+0000","last_unstale":"2024-05-09T08:27:56.780958+0000","last_undegraded":"2024-05-09T08:27:56.780958+0000","last_fullsized":"2024-05-09T08:27:56.780958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:19:33.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909787+0000","last_change":"2024-05-09T08:27:55.804770+0000","last_active":"2024-05-09T08:27:57.909787+0000","last_peered":"2024-05-09T08:27:57.909787+0000","last_clean":"2024-05-09T08:27:57.909787+0000","last_became_active":"2024-05-09T08:27:55.803486+0000","last_became_peered":"2024-05-09T08:27:55.803486+0000","last_unstale":"2024-05-09T08:27:57.909787+0000","last_undegraded":"2024-05-09T08:27:57.909787+0000","last_fullsized":"2024-05-09T08:27:57.909787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:28.641874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910780+0000","last_change":"2024-05-09T08:27:55.826032+0000","last_active":"2024-05-09T08:27:57.910780+0000","last_peered":"2024-05-09T08:27:57.910780+0000","last_clean":"2024-05-09T08:27:57.910780+0000","last_became_active":"2024-05-09T08:27:55.825510+0000","last_became_peered":"2024-05-09T08:27:55.825510+0000","last_unstale":"2024-05-09T08:27:57.910780+0000","last_undegraded":"2024-05-09T08:27:57.910780+0000","last_fullsized":"2024-05-09T08:27:57.910780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:25.691361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911177+0000","last_change":"2024-05-09T08:27:55.816178+0000","last_active":"2024-05-09T08:27:57.911177+0000","last_peered":"2024-05-09T08:27:57.911177+0000","last_clean":"2024-05-09T08:27:57.911177+0000","last_became_active":"2024-05-09T08:27:55.814592+0000","last_became_peered":"2024-05-09T08:27:55.814592+0000","last_unstale":"2024-05-09T08:27:57.911177+0000","last_undegraded":"2024-05-09T08:27:57.911177+0000","last_fullsized":"2024-05-09T08:27:57.911177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:06.632314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906451+0000","last_change":"2024-05-09T08:27:55.827914+0000","last_active":"2024-05-09T08:27:57.906451+0000","last_peered":"2024-05-09T08:27:57.906451+0000","last_clean":"2024-05-09T08:27:57.906451+0000","last_became_active":"2024-05-09T08:27:55.827555+0000","last_became_peered":"2024-05-09T08:27:55.827555+0000","last_unstale":"2024-05-09T08:27:57.906451+0000","last_undegraded":"2024-05-09T08:27:57.906451+0000","last_fullsized":"2024-05-09T08:27:57.906451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:37.097461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907181+0000","last_change":"2024-05-09T08:27:55.828072+0000","last_active":"2024-05-09T08:27:57.907181+0000","last_peered":"2024-05-09T08:27:57.907181+0000","last_clean":"2024-05-09T08:27:57.907181+0000","last_became_active":"2024-05-09T08:27:55.827727+0000","last_became_peered":"2024-05-09T08:27:55.827727+0000","last_unstale":"2024-05-09T08:27:57.907181+0000","last_undegraded":"2024-05-09T08:27:57.907181+0000","last_fullsized":"2024-05-09T08:27:57.907181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:13.761560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.782409+0000","last_change":"2024-05-09T08:27:55.812506+0000","last_active":"2024-05-09T08:27:56.782409+0000","last_peered":"2024-05-09T08:27:56.782409+0000","last_clean":"2024-05-09T08:27:56.782409+0000","last_became_active":"2024-05-09T08:27:55.812071+0000","last_became_peered":"2024-05-09T08:27:55.812071+0000","last_unstale":"2024-05-09T08:27:56.782409+0000","last_undegraded":"2024-05-09T08:27:56.782409+0000","last_fullsized":"2024-05-09T08:27:56.782409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:53.318140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780716+0000","last_change":"2024-05-09T08:27:55.817660+0000","last_active":"2024-05-09T08:27:56.780716+0000","last_peered":"2024-05-09T08:27:56.780716+0000","last_clean":"2024-05-09T08:27:56.780716+0000","last_became_active":"2024-05-09T08:27:55.817357+0000","last_became_peered":"2024-05-09T08:27:55.817357+0000","last_unstale":"2024-05-09T08:27:56.780716+0000","last_undegraded":"2024-05-09T08:27:56.780716+0000","last_fullsized":"2024-05-09T08:27:56.780716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:05.489063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270532+0000","last_change":"2024-05-09T08:27:54.784177+0000","last_active":"2024-05-09T08:28:00.270532+0000","last_peered":"2024-05-09T08:28:00.270532+0000","last_clean":"2024-05-09T08:28:00.270532+0000","last_became_active":"2024-05-09T08:27:54.780982+0000","last_became_peered":"2024-05-09T08:27:54.780982+0000","last_unstale":"2024-05-09T08:28:00.270532+0000","last_undegraded":"2024-05-09T08:28:00.270532+0000","last_fullsized":"2024-05-09T08:28:00.270532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:17.668694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226734+0000","last_change":"2024-05-09T08:27:55.825495+0000","last_active":"2024-05-09T08:27:57.226734+0000","last_peered":"2024-05-09T08:27:57.226734+0000","last_clean":"2024-05-09T08:27:57.226734+0000","last_became_active":"2024-05-09T08:27:55.821680+0000","last_became_peered":"2024-05-09T08:27:55.821680+0000","last_unstale":"2024-05-09T08:27:57.226734+0000","last_undegraded":"2024-05-09T08:27:57.226734+0000","last_fullsized":"2024-05-09T08:27:57.226734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:06.128200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:58.704233+0000","last_change":"2024-05-09T08:27:54.791362+0000","last_active":"2024-05-09T08:27:58.704233+0000","last_peered":"2024-05-09T08:27:58.704233+0000","last_clean":"2024-05-09T08:27:58.704233+0000","last_became_active":"2024-05-09T08:27:54.791050+0000","last_became_peered":"2024-05-09T08:27:54.791050+0000","last_unstale":"2024-05-09T08:27:58.704233+0000","last_undegraded":"2024-05-09T08:27:58.704233+0000","last_fullsized":"2024-05-09T08:27:58.704233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:11.821907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912541+0000","last_change":"2024-05-09T08:27:55.820278+0000","last_active":"2024-05-09T08:27:57.912541+0000","last_peered":"2024-05-09T08:27:57.912541+0000","last_clean":"2024-05-09T08:27:57.912541+0000","last_became_active":"2024-05-09T08:27:55.819861+0000","last_became_peered":"2024-05-09T08:27:55.819861+0000","last_unstale":"2024-05-09T08:27:57.912541+0000","last_undegraded":"2024-05-09T08:27:57.912541+0000","last_fullsized":"2024-05-09T08:27:57.912541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:29.541259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.267857+0000","last_change":"2024-05-09T08:27:54.771453+0000","last_active":"2024-05-09T08:28:00.267857+0000","last_peered":"2024-05-09T08:28:00.267857+0000","last_clean":"2024-05-09T08:28:00.267857+0000","last_became_active":"2024-05-09T08:27:54.770746+0000","last_became_peered":"2024-05-09T08:27:54.770746+0000","last_unstale":"2024-05-09T08:28:00.267857+0000","last_undegraded":"2024-05-09T08:28:00.267857+0000","last_fullsized":"2024-05-09T08:28:00.267857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:18.823562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906537+0000","last_change":"2024-05-09T08:27:55.803341+0000","last_active":"2024-05-09T08:27:57.906537+0000","last_peered":"2024-05-09T08:27:57.906537+0000","last_clean":"2024-05-09T08:27:57.906537+0000","last_became_active":"2024-05-09T08:27:55.802945+0000","last_became_peered":"2024-05-09T08:27:55.802945+0000","last_unstale":"2024-05-09T08:27:57.906537+0000","last_undegraded":"2024-05-09T08:27:57.906537+0000","last_fullsized":"2024-05-09T08:27:57.906537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:24.346139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225059+0000","last_change":"2024-05-09T08:27:54.769140+0000","last_active":"2024-05-09T08:27:57.225059+0000","last_peered":"2024-05-09T08:27:57.225059+0000","last_clean":"2024-05-09T08:27:57.225059+0000","last_became_active":"2024-05-09T08:27:54.768615+0000","last_became_peered":"2024-05-09T08:27:54.768615+0000","last_unstale":"2024-05-09T08:27:57.225059+0000","last_undegraded":"2024-05-09T08:27:57.225059+0000","last_fullsized":"2024-05-09T08:27:57.225059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:10.016070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907255+0000","last_change":"2024-05-09T08:27:55.806385+0000","last_active":"2024-05-09T08:27:57.907255+0000","last_peered":"2024-05-09T08:27:57.907255+0000","last_clean":"2024-05-09T08:27:57.907255+0000","last_became_active":"2024-05-09T08:27:55.806031+0000","last_became_peered":"2024-05-09T08:27:55.806031+0000","last_unstale":"2024-05-09T08:27:57.907255+0000","last_undegraded":"2024-05-09T08:27:57.907255+0000","last_fullsized":"2024-05-09T08:27:57.907255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:17.326277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.282887+0000","last_change":"2024-05-09T08:27:54.769148+0000","last_active":"2024-05-09T08:28:02.282887+0000","last_peered":"2024-05-09T08:28:02.282887+0000","last_clean":"2024-05-09T08:28:02.282887+0000","last_became_active":"2024-05-09T08:27:54.768814+0000","last_became_peered":"2024-05-09T08:27:54.768814+0000","last_unstale":"2024-05-09T08:28:02.282887+0000","last_undegraded":"2024-05-09T08:28:02.282887+0000","last_fullsized":"2024-05-09T08:28:02.282887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:08.276294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224830+0000","last_change":"2024-05-09T08:27:55.812884+0000","last_active":"2024-05-09T08:27:57.224830+0000","last_peered":"2024-05-09T08:27:57.224830+0000","last_clean":"2024-05-09T08:27:57.224830+0000","last_became_active":"2024-05-09T08:27:55.812515+0000","last_became_peered":"2024-05-09T08:27:55.812515+0000","last_unstale":"2024-05-09T08:27:57.224830+0000","last_undegraded":"2024-05-09T08:27:57.224830+0000","last_fullsized":"2024-05-09T08:27:57.224830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:59.211293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.268419+0000","last_change":"2024-05-09T08:27:54.764806+0000","last_active":"2024-05-09T08:28:00.268419+0000","last_peered":"2024-05-09T08:28:00.268419+0000","last_clean":"2024-05-09T08:28:00.268419+0000","last_became_active":"2024-05-09T08:27:54.764325+0000","last_became_peered":"2024-05-09T08:27:54.764325+0000","last_unstale":"2024-05-09T08:28:00.268419+0000","last_undegraded":"2024-05-09T08:28:00.268419+0000","last_fullsized":"2024-05-09T08:28:00.268419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:11.820240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907385+0000","last_change":"2024-05-09T08:27:55.822481+0000","last_active":"2024-05-09T08:27:57.907385+0000","last_peered":"2024-05-09T08:27:57.907385+0000","last_clean":"2024-05-09T08:27:57.907385+0000","last_became_active":"2024-05-09T08:27:55.821180+0000","last_became_peered":"2024-05-09T08:27:55.821180+0000","last_unstale":"2024-05-09T08:27:57.907385+0000","last_undegraded":"2024-05-09T08:27:57.907385+0000","last_fullsized":"2024-05-09T08:27:57.907385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:33:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.271850+0000","last_change":"2024-05-09T08:27:54.771263+0000","last_active":"2024-05-09T08:28:00.271850+0000","last_peered":"2024-05-09T08:28:00.271850+0000","last_clean":"2024-05-09T08:28:00.271850+0000","last_became_active":"2024-05-09T08:27:54.770719+0000","last_became_peered":"2024-05-09T08:27:54.770719+0000","last_unstale":"2024-05-09T08:28:00.271850+0000","last_undegraded":"2024-05-09T08:28:00.271850+0000","last_fullsized":"2024-05-09T08:28:00.271850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:08.725081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226429+0000","last_change":"2024-05-09T08:27:55.821493+0000","last_active":"2024-05-09T08:27:57.226429+0000","last_peered":"2024-05-09T08:27:57.226429+0000","last_clean":"2024-05-09T08:27:57.226429+0000","last_became_active":"2024-05-09T08:27:55.818585+0000","last_became_peered":"2024-05-09T08:27:55.818585+0000","last_unstale":"2024-05-09T08:27:57.226429+0000","last_undegraded":"2024-05-09T08:27:57.226429+0000","last_fullsized":"2024-05-09T08:27:57.226429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:56.319691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906205+0000","last_change":"2024-05-09T08:27:54.769856+0000","last_active":"2024-05-09T08:27:57.906205+0000","last_peered":"2024-05-09T08:27:57.906205+0000","last_clean":"2024-05-09T08:27:57.906205+0000","last_became_active":"2024-05-09T08:27:54.769548+0000","last_became_peered":"2024-05-09T08:27:54.769548+0000","last_unstale":"2024-05-09T08:27:57.906205+0000","last_undegraded":"2024-05-09T08:27:57.906205+0000","last_fullsized":"2024-05-09T08:27:57.906205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:57.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.781667+0000","last_change":"2024-05-09T08:27:55.822063+0000","last_active":"2024-05-09T08:27:56.781667+0000","last_peered":"2024-05-09T08:27:56.781667+0000","last_clean":"2024-05-09T08:27:56.781667+0000","last_became_active":"2024-05-09T08:27:55.821624+0000","last_became_peered":"2024-05-09T08:27:55.821624+0000","last_unstale":"2024-05-09T08:27:56.781667+0000","last_undegraded":"2024-05-09T08:27:56.781667+0000","last_fullsized":"2024-05-09T08:27:56.781667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:14.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.259800+0000","last_change":"2024-05-09T08:27:54.767639+0000","last_active":"2024-05-09T08:27:59.259800+0000","last_peered":"2024-05-09T08:27:59.259800+0000","last_clean":"2024-05-09T08:27:59.259800+0000","last_became_active":"2024-05-09T08:27:54.767284+0000","last_became_peered":"2024-05-09T08:27:54.767284+0000","last_unstale":"2024-05-09T08:27:59.259800+0000","last_undegraded":"2024-05-09T08:27:59.259800+0000","last_fullsized":"2024-05-09T08:27:59.259800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:07.110319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906933+0000","last_change":"2024-05-09T08:27:55.817050+0000","last_active":"2024-05-09T08:27:57.906933+0000","last_peered":"2024-05-09T08:27:57.906933+0000","last_clean":"2024-05-09T08:27:57.906933+0000","last_became_active":"2024-05-09T08:27:55.816655+0000","last_became_peered":"2024-05-09T08:27:55.816655+0000","last_unstale":"2024-05-09T08:27:57.906933+0000","last_undegraded":"2024-05-09T08:27:57.906933+0000","last_fullsized":"2024-05-09T08:27:57.906933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:06.132482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258793+0000","last_change":"2024-05-09T08:27:54.770449+0000","last_active":"2024-05-09T08:27:59.258793+0000","last_peered":"2024-05-09T08:27:59.258793+0000","last_clean":"2024-05-09T08:27:59.258793+0000","last_became_active":"2024-05-09T08:27:54.769382+0000","last_became_peered":"2024-05-09T08:27:54.769382+0000","last_unstale":"2024-05-09T08:27:59.258793+0000","last_undegraded":"2024-05-09T08:27:59.258793+0000","last_fullsized":"2024-05-09T08:27:59.258793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:47.108749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912376+0000","last_change":"2024-05-09T08:27:55.831416+0000","last_active":"2024-05-09T08:27:57.912376+0000","last_peered":"2024-05-09T08:27:57.912376+0000","last_clean":"2024-05-09T08:27:57.912376+0000","last_became_active":"2024-05-09T08:27:55.831011+0000","last_became_peered":"2024-05-09T08:27:55.831011+0000","last_unstale":"2024-05-09T08:27:57.912376+0000","last_undegraded":"2024-05-09T08:27:57.912376+0000","last_fullsized":"2024-05-09T08:27:57.912376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:23.820425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258788+0000","last_change":"2024-05-09T08:27:54.776737+0000","last_active":"2024-05-09T08:27:59.258788+0000","last_peered":"2024-05-09T08:27:59.258788+0000","last_clean":"2024-05-09T08:27:59.258788+0000","last_became_active":"2024-05-09T08:27:54.776402+0000","last_became_peered":"2024-05-09T08:27:54.776402+0000","last_unstale":"2024-05-09T08:27:59.258788+0000","last_undegraded":"2024-05-09T08:27:59.258788+0000","last_fullsized":"2024-05-09T08:27:59.258788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:38.135159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906693+0000","last_change":"2024-05-09T08:27:55.810092+0000","last_active":"2024-05-09T08:27:57.906693+0000","last_peered":"2024-05-09T08:27:57.906693+0000","last_clean":"2024-05-09T08:27:57.906693+0000","last_became_active":"2024-05-09T08:27:55.809738+0000","last_became_peered":"2024-05-09T08:27:55.809738+0000","last_unstale":"2024-05-09T08:27:57.906693+0000","last_undegraded":"2024-05-09T08:27:57.906693+0000","last_fullsized":"2024-05-09T08:27:57.906693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:34.094014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:01.282643+0000","last_change":"2024-05-09T08:27:54.769608+0000","last_active":"2024-05-09T08:28:01.282643+0000","last_peered":"2024-05-09T08:28:01.282643+0000","last_clean":"2024-05-09T08:28:01.282643+0000","last_became_active":"2024-05-09T08:27:54.769286+0000","last_became_peered":"2024-05-09T08:27:54.769286+0000","last_unstale":"2024-05-09T08:28:01.282643+0000","last_undegraded":"2024-05-09T08:28:01.282643+0000","last_fullsized":"2024-05-09T08:28:01.282643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:40.620602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225059+0000","last_change":"2024-05-09T08:27:55.800489+0000","last_active":"2024-05-09T08:27:57.225059+0000","last_peered":"2024-05-09T08:27:57.225059+0000","last_clean":"2024-05-09T08:27:57.225059+0000","last_became_active":"2024-05-09T08:27:55.800145+0000","last_became_peered":"2024-05-09T08:27:55.800145+0000","last_unstale":"2024-05-09T08:27:57.225059+0000","last_undegraded":"2024-05-09T08:27:57.225059+0000","last_fullsized":"2024-05-09T08:27:57.225059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:47.877630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.271696+0000","last_change":"2024-05-09T08:27:54.779179+0000","last_active":"2024-05-09T08:28:00.271696+0000","last_peered":"2024-05-09T08:28:00.271696+0000","last_clean":"2024-05-09T08:28:00.271696+0000","last_became_active":"2024-05-09T08:27:54.778793+0000","last_became_peered":"2024-05-09T08:27:54.778793+0000","last_unstale":"2024-05-09T08:28:00.271696+0000","last_undegraded":"2024-05-09T08:28:00.271696+0000","last_fullsized":"2024-05-09T08:28:00.271696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:56:42.414959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909073+0000","last_change":"2024-05-09T08:27:55.830500+0000","last_active":"2024-05-09T08:27:57.909073+0000","last_peered":"2024-05-09T08:27:57.909073+0000","last_clean":"2024-05-09T08:27:57.909073+0000","last_became_active":"2024-05-09T08:27:55.830044+0000","last_became_peered":"2024-05-09T08:27:55.830044+0000","last_unstale":"2024-05-09T08:27:57.909073+0000","last_undegraded":"2024-05-09T08:27:57.909073+0000","last_fullsized":"2024-05-09T08:27:57.909073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:15.616303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.260714+0000","last_change":"2024-05-09T08:27:54.786607+0000","last_active":"2024-05-09T08:27:59.260714+0000","last_peered":"2024-05-09T08:27:59.260714+0000","last_clean":"2024-05-09T08:27:59.260714+0000","last_became_active":"2024-05-09T08:27:54.786286+0000","last_became_peered":"2024-05-09T08:27:54.786286+0000","last_unstale":"2024-05-09T08:27:59.260714+0000","last_undegraded":"2024-05-09T08:27:59.260714+0000","last_fullsized":"2024-05-09T08:27:59.260714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.833237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387752+0000","last_change":"2024-05-09T08:27:55.809022+0000","last_active":"2024-05-09T08:27:57.387752+0000","last_peered":"2024-05-09T08:27:57.387752+0000","last_clean":"2024-05-09T08:27:57.387752+0000","last_became_active":"2024-05-09T08:27:55.808666+0000","last_became_peered":"2024-05-09T08:27:55.808666+0000","last_unstale":"2024-05-09T08:27:57.387752+0000","last_undegraded":"2024-05-09T08:27:57.387752+0000","last_fullsized":"2024-05-09T08:27:57.387752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:18.339576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.287215+0000","last_change":"2024-05-09T08:27:54.769166+0000","last_active":"2024-05-09T08:28:02.287215+0000","last_peered":"2024-05-09T08:28:02.287215+0000","last_clean":"2024-05-09T08:28:02.287215+0000","last_became_active":"2024-05-09T08:27:54.768794+0000","last_became_peered":"2024-05-09T08:27:54.768794+0000","last_unstale":"2024-05-09T08:28:02.287215+0000","last_undegraded":"2024-05-09T08:28:02.287215+0000","last_fullsized":"2024-05-09T08:28:02.287215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.222388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387305+0000","last_change":"2024-05-09T08:27:55.806685+0000","last_active":"2024-05-09T08:27:57.387305+0000","last_peered":"2024-05-09T08:27:57.387305+0000","last_clean":"2024-05-09T08:27:57.387305+0000","last_became_active":"2024-05-09T08:27:55.804641+0000","last_became_peered":"2024-05-09T08:27:55.804641+0000","last_unstale":"2024-05-09T08:27:57.387305+0000","last_undegraded":"2024-05-09T08:27:57.387305+0000","last_fullsized":"2024-05-09T08:27:57.387305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:40.307764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270776+0000","last_change":"2024-05-09T08:27:54.782524+0000","last_active":"2024-05-09T08:28:00.270776+0000","last_peered":"2024-05-09T08:28:00.270776+0000","last_clean":"2024-05-09T08:28:00.270776+0000","last_became_active":"2024-05-09T08:27:54.782220+0000","last_became_peered":"2024-05-09T08:27:54.782220+0000","last_unstale":"2024-05-09T08:28:00.270776+0000","last_undegraded":"2024-05-09T08:28:00.270776+0000","last_fullsized":"2024-05-09T08:28:00.270776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:16:58.705682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906487+0000","last_change":"2024-05-09T08:27:55.806390+0000","last_active":"2024-05-09T08:27:57.906487+0000","last_peered":"2024-05-09T08:27:57.906487+0000","last_clean":"2024-05-09T08:27:57.906487+0000","last_became_active":"2024-05-09T08:27:55.806027+0000","last_became_peered":"2024-05-09T08:27:55.806027+0000","last_unstale":"2024-05-09T08:27:57.906487+0000","last_undegraded":"2024-05-09T08:27:57.906487+0000","last_fullsized":"2024-05-09T08:27:57.906487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:58.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225231+0000","last_change":"2024-05-09T08:27:54.766679+0000","last_active":"2024-05-09T08:27:57.225231+0000","last_peered":"2024-05-09T08:27:57.225231+0000","last_clean":"2024-05-09T08:27:57.225231+0000","last_became_active":"2024-05-09T08:27:54.766318+0000","last_became_peered":"2024-05-09T08:27:54.766318+0000","last_unstale":"2024-05-09T08:27:57.225231+0000","last_undegraded":"2024-05-09T08:27:57.225231+0000","last_fullsized":"2024-05-09T08:27:57.225231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:38.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906734+0000","last_change":"2024-05-09T08:27:55.804379+0000","last_active":"2024-05-09T08:27:57.906734+0000","last_peered":"2024-05-09T08:27:57.906734+0000","last_clean":"2024-05-09T08:27:57.906734+0000","last_became_active":"2024-05-09T08:27:55.804046+0000","last_became_peered":"2024-05-09T08:27:55.804046+0000","last_unstale":"2024-05-09T08:27:57.906734+0000","last_undegraded":"2024-05-09T08:27:57.906734+0000","last_fullsized":"2024-05-09T08:27:57.906734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:43.735487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.782028+0000","last_change":"2024-05-09T08:27:54.785336+0000","last_active":"2024-05-09T08:27:56.782028+0000","last_peered":"2024-05-09T08:27:56.782028+0000","last_clean":"2024-05-09T08:27:56.782028+0000","last_became_active":"2024-05-09T08:27:54.784581+0000","last_became_peered":"2024-05-09T08:27:54.784581+0000","last_unstale":"2024-05-09T08:27:56.782028+0000","last_undegraded":"2024-05-09T08:27:56.782028+0000","last_fullsized":"2024-05-09T08:27:56.782028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:40.071747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978840+0000","last_change":"2024-05-09T08:27:55.821512+0000","last_active":"2024-05-09T08:27:56.978840+0000","last_peered":"2024-05-09T08:27:56.978840+0000","last_clean":"2024-05-09T08:27:56.978840+0000","last_became_active":"2024-05-09T08:27:55.821162+0000","last_became_peered":"2024-05-09T08:27:55.821162+0000","last_unstale":"2024-05-09T08:27:56.978840+0000","last_undegraded":"2024-05-09T08:27:56.978840+0000","last_fullsized":"2024-05-09T08:27:56.978840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:47.377452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387417+0000","last_change":"2024-05-09T08:27:54.771528+0000","last_active":"2024-05-09T08:27:57.387417+0000","last_peered":"2024-05-09T08:27:57.387417+0000","last_clean":"2024-05-09T08:27:57.387417+0000","last_became_active":"2024-05-09T08:27:54.771133+0000","last_became_peered":"2024-05-09T08:27:54.771133+0000","last_unstale":"2024-05-09T08:27:57.387417+0000","last_undegraded":"2024-05-09T08:27:57.387417+0000","last_fullsized":"2024-05-09T08:27:57.387417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:22.948800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978312+0000","last_change":"2024-05-09T08:27:55.824954+0000","last_active":"2024-05-09T08:27:56.978312+0000","last_peered":"2024-05-09T08:27:56.978312+0000","last_clean":"2024-05-09T08:27:56.978312+0000","last_became_active":"2024-05-09T08:27:55.824578+0000","last_became_peered":"2024-05-09T08:27:55.824578+0000","last_unstale":"2024-05-09T08:27:56.978312+0000","last_undegraded":"2024-05-09T08:27:56.978312+0000","last_fullsized":"2024-05-09T08:27:56.978312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:35:22.790193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906602+0000","last_change":"2024-05-09T08:27:55.804508+0000","last_active":"2024-05-09T08:27:57.906602+0000","last_peered":"2024-05-09T08:27:57.906602+0000","last_clean":"2024-05-09T08:27:57.906602+0000","last_became_active":"2024-05-09T08:27:55.804190+0000","last_became_peered":"2024-05-09T08:27:55.804190+0000","last_unstale":"2024-05-09T08:27:57.906602+0000","last_undegraded":"2024-05-09T08:27:57.906602+0000","last_fullsized":"2024-05-09T08:27:57.906602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:24.232140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.285504+0000","last_change":"2024-05-09T08:27:54.780486+0000","last_active":"2024-05-09T08:28:02.285504+0000","last_peered":"2024-05-09T08:28:02.285504+0000","last_clean":"2024-05-09T08:28:02.285504+0000","last_became_active":"2024-05-09T08:27:54.777349+0000","last_became_peered":"2024-05-09T08:27:54.777349+0000","last_unstale":"2024-05-09T08:28:02.285504+0000","last_undegraded":"2024-05-09T08:28:02.285504+0000","last_fullsized":"2024-05-09T08:28:02.285504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:14.729741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226851+0000","last_change":"2024-05-09T08:27:55.825469+0000","last_active":"2024-05-09T08:27:57.226851+0000","last_peered":"2024-05-09T08:27:57.226851+0000","last_clean":"2024-05-09T08:27:57.226851+0000","last_became_active":"2024-05-09T08:27:55.825103+0000","last_became_peered":"2024-05-09T08:27:55.825103+0000","last_unstale":"2024-05-09T08:27:57.226851+0000","last_undegraded":"2024-05-09T08:27:57.226851+0000","last_fullsized":"2024-05-09T08:27:57.226851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:53.136741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:01.278178+0000","last_change":"2024-05-09T08:27:54.774194+0000","last_active":"2024-05-09T08:28:01.278178+0000","last_peered":"2024-05-09T08:28:01.278178+0000","last_clean":"2024-05-09T08:28:01.278178+0000","last_became_active":"2024-05-09T08:27:54.773678+0000","last_became_peered":"2024-05-09T08:27:54.773678+0000","last_unstale":"2024-05-09T08:28:01.278178+0000","last_undegraded":"2024-05-09T08:28:01.278178+0000","last_fullsized":"2024-05-09T08:28:01.278178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:23.199301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387185+0000","last_change":"2024-05-09T08:26:34.966221+0000","last_active":"2024-05-09T08:27:57.387185+0000","last_peered":"2024-05-09T08:27:57.387185+0000","last_clean":"2024-05-09T08:27:57.387185+0000","last_became_active":"2024-05-09T08:26:34.965800+0000","last_became_peered":"2024-05-09T08:26:34.965800+0000","last_unstale":"2024-05-09T08:27:57.387185+0000","last_undegraded":"2024-05-09T08:27:57.387185+0000","last_fullsized":"2024-05-09T08:27:57.387185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_clean_scrub_stamp":"2024-05-09T08:22:41.596226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:10.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909184+0000","last_change":"2024-05-09T08:27:55.805417+0000","last_active":"2024-05-09T08:27:57.909184+0000","last_peered":"2024-05-09T08:27:57.909184+0000","last_clean":"2024-05-09T08:27:57.909184+0000","last_became_active":"2024-05-09T08:27:55.805025+0000","last_became_peered":"2024-05-09T08:27:55.805025+0000","last_unstale":"2024-05-09T08:27:57.909184+0000","last_undegraded":"2024-05-09T08:27:57.909184+0000","last_fullsized":"2024-05-09T08:27:57.909184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:35.442848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.267533+0000","last_change":"2024-05-09T08:27:54.769276+0000","last_active":"2024-05-09T08:28:00.267533+0000","last_peered":"2024-05-09T08:28:00.267533+0000","last_clean":"2024-05-09T08:28:00.267533+0000","last_became_active":"2024-05-09T08:27:54.768650+0000","last_became_peered":"2024-05-09T08:27:54.768650+0000","last_unstale":"2024-05-09T08:28:00.267533+0000","last_undegraded":"2024-05-09T08:28:00.267533+0000","last_fullsized":"2024-05-09T08:28:00.267533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:32.480610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225151+0000","last_change":"2024-05-09T08:27:54.782809+0000","last_active":"2024-05-09T08:27:57.225151+0000","last_peered":"2024-05-09T08:27:57.225151+0000","last_clean":"2024-05-09T08:27:57.225151+0000","last_became_active":"2024-05-09T08:27:54.782452+0000","last_became_peered":"2024-05-09T08:27:54.782452+0000","last_unstale":"2024-05-09T08:27:57.225151+0000","last_undegraded":"2024-05-09T08:27:57.225151+0000","last_fullsized":"2024-05-09T08:27:57.225151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:42.371573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909128+0000","last_change":"2024-05-09T08:27:55.814683+0000","last_active":"2024-05-09T08:27:57.909128+0000","last_peered":"2024-05-09T08:27:57.909128+0000","last_clean":"2024-05-09T08:27:57.909128+0000","last_became_active":"2024-05-09T08:27:55.813788+0000","last_became_peered":"2024-05-09T08:27:55.813788+0000","last_unstale":"2024-05-09T08:27:57.909128+0000","last_undegraded":"2024-05-09T08:27:57.909128+0000","last_fullsized":"2024-05-09T08:27:57.909128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:45.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:03.225474+0000","last_change":"2024-05-09T08:27:54.788393+0000","last_active":"2024-05-09T08:28:03.225474+0000","last_peered":"2024-05-09T08:28:03.225474+0000","last_clean":"2024-05-09T08:28:03.225474+0000","last_became_active":"2024-05-09T08:27:54.788031+0000","last_became_peered":"2024-05-09T08:27:54.788031+0000","last_unstale":"2024-05-09T08:28:03.225474+0000","last_undegraded":"2024-05-09T08:28:03.225474+0000","last_fullsized":"2024-05-09T08:28:03.225474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:26.451385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227322+0000","last_change":"2024-05-09T08:27:55.812391+0000","last_active":"2024-05-09T08:27:57.227322+0000","last_peered":"2024-05-09T08:27:57.227322+0000","last_clean":"2024-05-09T08:27:57.227322+0000","last_became_active":"2024-05-09T08:27:55.811954+0000","last_became_peered":"2024-05-09T08:27:55.811954+0000","last_unstale":"2024-05-09T08:27:57.227322+0000","last_undegraded":"2024-05-09T08:27:57.227322+0000","last_fullsized":"2024-05-09T08:27:57.227322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:38.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258030+0000","last_change":"2024-05-09T08:27:54.775693+0000","last_active":"2024-05-09T08:27:59.258030+0000","last_peered":"2024-05-09T08:27:59.258030+0000","last_clean":"2024-05-09T08:27:59.258030+0000","last_became_active":"2024-05-09T08:27:54.775198+0000","last_became_peered":"2024-05-09T08:27:54.775198+0000","last_unstale":"2024-05-09T08:27:59.258030+0000","last_undegraded":"2024-05-09T08:27:59.258030+0000","last_fullsized":"2024-05-09T08:27:59.258030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:21:18.672744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912231+0000","last_change":"2024-05-09T08:27:55.822832+0000","last_active":"2024-05-09T08:27:57.912231+0000","last_peered":"2024-05-09T08:27:57.912231+0000","last_clean":"2024-05-09T08:27:57.912231+0000","last_became_active":"2024-05-09T08:27:55.822391+0000","last_became_peered":"2024-05-09T08:27:55.822391+0000","last_unstale":"2024-05-09T08:27:57.912231+0000","last_undegraded":"2024-05-09T08:27:57.912231+0000","last_fullsized":"2024-05-09T08:27:57.912231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:00.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258927+0000","last_change":"2024-05-09T08:27:54.791027+0000","last_active":"2024-05-09T08:27:59.258927+0000","last_peered":"2024-05-09T08:27:59.258927+0000","last_clean":"2024-05-09T08:27:59.258927+0000","last_became_active":"2024-05-09T08:27:54.790726+0000","last_became_peered":"2024-05-09T08:27:54.790726+0000","last_unstale":"2024-05-09T08:27:59.258927+0000","last_undegraded":"2024-05-09T08:27:59.258927+0000","last_fullsized":"2024-05-09T08:27:59.258927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:37.795710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224733+0000","last_change":"2024-05-09T08:27:55.799349+0000","last_active":"2024-05-09T08:27:57.224733+0000","last_peered":"2024-05-09T08:27:57.224733+0000","last_clean":"2024-05-09T08:27:57.224733+0000","last_became_active":"2024-05-09T08:27:55.798931+0000","last_became_peered":"2024-05-09T08:27:55.798931+0000","last_unstale":"2024-05-09T08:27:57.224733+0000","last_undegraded":"2024-05-09T08:27:57.224733+0000","last_fullsized":"2024-05-09T08:27:57.224733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:15.824884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.261110+0000","last_change":"2024-05-09T08:27:54.792088+0000","last_active":"2024-05-09T08:27:59.261110+0000","last_peered":"2024-05-09T08:27:59.261110+0000","last_clean":"2024-05-09T08:27:59.261110+0000","last_became_active":"2024-05-09T08:27:54.791526+0000","last_became_peered":"2024-05-09T08:27:54.791526+0000","last_unstale":"2024-05-09T08:27:59.261110+0000","last_undegraded":"2024-05-09T08:27:59.261110+0000","last_fullsized":"2024-05-09T08:27:59.261110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:11.494625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906651+0000","last_change":"2024-05-09T08:27:55.823258+0000","last_active":"2024-05-09T08:27:57.906651+0000","last_peered":"2024-05-09T08:27:57.906651+0000","last_clean":"2024-05-09T08:27:57.906651+0000","last_became_active":"2024-05-09T08:27:55.822923+0000","last_became_peered":"2024-05-09T08:27:55.822923+0000","last_unstale":"2024-05-09T08:27:57.906651+0000","last_undegraded":"2024-05-09T08:27:57.906651+0000","last_fullsized":"2024-05-09T08:27:57.906651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:17.481997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780714+0000","last_change":"2024-05-09T08:27:54.778865+0000","last_active":"2024-05-09T08:27:56.780714+0000","last_peered":"2024-05-09T08:27:56.780714+0000","last_clean":"2024-05-09T08:27:56.780714+0000","last_became_active":"2024-05-09T08:27:54.778494+0000","last_became_peered":"2024-05-09T08:27:54.778494+0000","last_unstale":"2024-05-09T08:27:56.780714+0000","last_undegraded":"2024-05-09T08:27:56.780714+0000","last_fullsized":"2024-05-09T08:27:56.780714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:19.632591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226515+0000","last_change":"2024-05-09T08:27:55.829385+0000","last_active":"2024-05-09T08:27:57.226515+0000","last_peered":"2024-05-09T08:27:57.226515+0000","last_clean":"2024-05-09T08:27:57.226515+0000","last_became_active":"2024-05-09T08:27:55.829045+0000","last_became_peered":"2024-05-09T08:27:55.829045+0000","last_unstale":"2024-05-09T08:27:57.226515+0000","last_undegraded":"2024-05-09T08:27:57.226515+0000","last_fullsized":"2024-05-09T08:27:57.226515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.171868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.257803+0000","last_change":"2024-05-09T08:27:54.770934+0000","last_active":"2024-05-09T08:27:59.257803+0000","last_peered":"2024-05-09T08:27:59.257803+0000","last_clean":"2024-05-09T08:27:59.257803+0000","last_became_active":"2024-05-09T08:27:54.770597+0000","last_became_peered":"2024-05-09T08:27:54.770597+0000","last_unstale":"2024-05-09T08:27:59.257803+0000","last_undegraded":"2024-05-09T08:27:59.257803+0000","last_fullsized":"2024-05-09T08:27:59.257803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:20.722491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387712+0000","last_change":"2024-05-09T08:27:55.804210+0000","last_active":"2024-05-09T08:27:57.387712+0000","last_peered":"2024-05-09T08:27:57.387712+0000","last_clean":"2024-05-09T08:27:57.387712+0000","last_became_active":"2024-05-09T08:27:55.803870+0000","last_became_peered":"2024-05-09T08:27:55.803870+0000","last_unstale":"2024-05-09T08:27:57.387712+0000","last_undegraded":"2024-05-09T08:27:57.387712+0000","last_fullsized":"2024-05-09T08:27:57.387712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:46.593084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270724+0000","last_change":"2024-05-09T08:27:54.775110+0000","last_active":"2024-05-09T08:28:00.270724+0000","last_peered":"2024-05-09T08:28:00.270724+0000","last_clean":"2024-05-09T08:28:00.270724+0000","last_became_active":"2024-05-09T08:27:54.774812+0000","last_became_peered":"2024-05-09T08:27:54.774812+0000","last_unstale":"2024-05-09T08:28:00.270724+0000","last_undegraded":"2024-05-09T08:28:00.270724+0000","last_fullsized":"2024-05-09T08:28:00.270724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:51.955284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387809+0000","last_change":"2024-05-09T08:27:55.804766+0000","last_active":"2024-05-09T08:27:57.387809+0000","last_peered":"2024-05-09T08:27:57.387809+0000","last_clean":"2024-05-09T08:27:57.387809+0000","last_became_active":"2024-05-09T08:27:55.803711+0000","last_became_peered":"2024-05-09T08:27:55.803711+0000","last_unstale":"2024-05-09T08:27:57.387809+0000","last_undegraded":"2024-05-09T08:27:57.387809+0000","last_fullsized":"2024-05-09T08:27:57.387809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:46.683112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907030+0000","last_change":"2024-05-09T08:27:54.786594+0000","last_active":"2024-05-09T08:27:57.907030+0000","last_peered":"2024-05-09T08:27:57.907030+0000","last_clean":"2024-05-09T08:27:57.907030+0000","last_became_active":"2024-05-09T08:27:54.786329+0000","last_became_peered":"2024-05-09T08:27:54.786329+0000","last_unstale":"2024-05-09T08:27:57.907030+0000","last_undegraded":"2024-05-09T08:27:57.907030+0000","last_fullsized":"2024-05-09T08:27:57.907030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:09:43.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.389719+0000","last_change":"2024-05-09T08:27:55.825679+0000","last_active":"2024-05-09T08:27:59.389719+0000","last_peered":"2024-05-09T08:27:59.389719+0000","last_clean":"2024-05-09T08:27:59.389719+0000","last_became_active":"2024-05-09T08:27:55.825294+0000","last_became_peered":"2024-05-09T08:27:55.825294+0000","last_unstale":"2024-05-09T08:27:59.389719+0000","last_undegraded":"2024-05-09T08:27:59.389719+0000","last_fullsized":"2024-05-09T08:27:59.389719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:34:07.114046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910694+0000","last_change":"2024-05-09T08:27:54.766960+0000","last_active":"2024-05-09T08:27:57.910694+0000","last_peered":"2024-05-09T08:27:57.910694+0000","last_clean":"2024-05-09T08:27:57.910694+0000","last_became_active":"2024-05-09T08:27:54.765693+0000","last_became_peered":"2024-05-09T08:27:54.765693+0000","last_unstale":"2024-05-09T08:27:57.910694+0000","last_undegraded":"2024-05-09T08:27:57.910694+0000","last_fullsized":"2024-05-09T08:27:57.910694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:29:50.080788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387721+0000","last_change":"2024-05-09T08:27:55.806802+0000","last_active":"2024-05-09T08:27:57.387721+0000","last_peered":"2024-05-09T08:27:57.387721+0000","last_clean":"2024-05-09T08:27:57.387721+0000","last_became_active":"2024-05-09T08:27:55.806421+0000","last_became_peered":"2024-05-09T08:27:55.806421+0000","last_unstale":"2024-05-09T08:27:57.387721+0000","last_undegraded":"2024-05-09T08:27:57.387721+0000","last_fullsized":"2024-05-09T08:27:57.387721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:45.673043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906660+0000","last_change":"2024-05-09T08:27:55.799360+0000","last_active":"2024-05-09T08:27:57.906660+0000","last_peered":"2024-05-09T08:27:57.906660+0000","last_clean":"2024-05-09T08:27:57.906660+0000","last_became_active":"2024-05-09T08:27:55.799018+0000","last_became_peered":"2024-05-09T08:27:55.799018+0000","last_unstale":"2024-05-09T08:27:57.906660+0000","last_undegraded":"2024-05-09T08:27:57.906660+0000","last_fullsized":"2024-05-09T08:27:57.906660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:54.767546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780412+0000","last_change":"2024-05-09T08:27:55.803385+0000","last_active":"2024-05-09T08:27:56.780412+0000","last_peered":"2024-05-09T08:27:56.780412+0000","last_clean":"2024-05-09T08:27:56.780412+0000","last_became_active":"2024-05-09T08:27:55.803031+0000","last_became_peered":"2024-05-09T08:27:55.803031+0000","last_unstale":"2024-05-09T08:27:56.780412+0000","last_undegraded":"2024-05-09T08:27:56.780412+0000","last_fullsized":"2024-05-09T08:27:56.780412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:47.750225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780405+0000","last_change":"2024-05-09T08:27:55.825634+0000","last_active":"2024-05-09T08:27:56.780405+0000","last_peered":"2024-05-09T08:27:56.780405+0000","last_clean":"2024-05-09T08:27:56.780405+0000","last_became_active":"2024-05-09T08:27:55.825330+0000","last_became_peered":"2024-05-09T08:27:55.825330+0000","last_unstale":"2024-05-09T08:27:56.780405+0000","last_undegraded":"2024-05-09T08:27:56.780405+0000","last_fullsized":"2024-05-09T08:27:56.780405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:21:18.166324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911087+0000","last_change":"2024-05-09T08:27:55.818372+0000","last_active":"2024-05-09T08:27:57.911087+0000","last_peered":"2024-05-09T08:27:57.911087+0000","last_clean":"2024-05-09T08:27:57.911087+0000","last_became_active":"2024-05-09T08:27:55.817982+0000","last_became_peered":"2024-05-09T08:27:55.817982+0000","last_unstale":"2024-05-09T08:27:57.911087+0000","last_undegraded":"2024-05-09T08:27:57.911087+0000","last_fullsized":"2024-05-09T08:27:57.911087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:16.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909769+0000","last_change":"2024-05-09T08:27:55.820509+0000","last_active":"2024-05-09T08:27:57.909769+0000","last_peered":"2024-05-09T08:27:57.909769+0000","last_clean":"2024-05-09T08:27:57.909769+0000","last_became_active":"2024-05-09T08:27:55.820135+0000","last_became_peered":"2024-05-09T08:27:55.820135+0000","last_unstale":"2024-05-09T08:27:57.909769+0000","last_undegraded":"2024-05-09T08:27:57.909769+0000","last_fullsized":"2024-05-09T08:27:57.909769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:28.691640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906864+0000","last_change":"2024-05-09T08:27:55.801038+0000","last_active":"2024-05-09T08:27:57.906864+0000","last_peered":"2024-05-09T08:27:57.906864+0000","last_clean":"2024-05-09T08:27:57.906864+0000","last_became_active":"2024-05-09T08:27:55.800700+0000","last_became_peered":"2024-05-09T08:27:55.800700+0000","last_unstale":"2024-05-09T08:27:57.906864+0000","last_undegraded":"2024-05-09T08:27:57.906864+0000","last_fullsized":"2024-05-09T08:27:57.906864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:44:42.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780866+0000","last_change":"2024-05-09T08:27:55.802266+0000","last_active":"2024-05-09T08:27:56.780866+0000","last_peered":"2024-05-09T08:27:56.780866+0000","last_clean":"2024-05-09T08:27:56.780866+0000","last_became_active":"2024-05-09T08:27:55.801670+0000","last_became_peered":"2024-05-09T08:27:55.801670+0000","last_unstale":"2024-05-09T08:27:56.780866+0000","last_undegraded":"2024-05-09T08:27:56.780866+0000","last_fullsized":"2024-05-09T08:27:56.780866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:53.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227177+0000","last_change":"2024-05-09T08:27:55.820107+0000","last_active":"2024-05-09T08:27:57.227177+0000","last_peered":"2024-05-09T08:27:57.227177+0000","last_clean":"2024-05-09T08:27:57.227177+0000","last_became_active":"2024-05-09T08:27:55.819748+0000","last_became_peered":"2024-05-09T08:27:55.819748+0000","last_unstale":"2024-05-09T08:27:57.227177+0000","last_undegraded":"2024-05-09T08:27:57.227177+0000","last_fullsized":"2024-05-09T08:27:57.227177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:08:04.311374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906430+0000","last_change":"2024-05-09T08:27:55.821625+0000","last_active":"2024-05-09T08:27:57.906430+0000","last_peered":"2024-05-09T08:27:57.906430+0000","last_clean":"2024-05-09T08:27:57.906430+0000","last_became_active":"2024-05-09T08:27:55.821291+0000","last_became_peered":"2024-05-09T08:27:55.821291+0000","last_unstale":"2024-05-09T08:27:57.906430+0000","last_undegraded":"2024-05-09T08:27:57.906430+0000","last_fullsized":"2024-05-09T08:27:57.906430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:40.017767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226118+0000","last_change":"2024-05-09T08:27:55.818212+0000","last_active":"2024-05-09T08:27:57.226118+0000","last_peered":"2024-05-09T08:27:57.226118+0000","last_clean":"2024-05-09T08:27:57.226118+0000","last_became_active":"2024-05-09T08:27:55.817874+0000","last_became_peered":"2024-05-09T08:27:55.817874+0000","last_unstale":"2024-05-09T08:27:57.226118+0000","last_undegraded":"2024-05-09T08:27:57.226118+0000","last_fullsized":"2024-05-09T08:27:57.226118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:20.189942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906627+0000","last_change":"2024-05-09T08:27:55.820128+0000","last_active":"2024-05-09T08:27:57.906627+0000","last_peered":"2024-05-09T08:27:57.906627+0000","last_clean":"2024-05-09T08:27:57.906627+0000","last_became_active":"2024-05-09T08:27:55.819717+0000","last_became_peered":"2024-05-09T08:27:55.819717+0000","last_unstale":"2024-05-09T08:27:57.906627+0000","last_undegraded":"2024-05-09T08:27:57.906627+0000","last_fullsized":"2024-05-09T08:27:57.906627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:48.324777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910235+0000","last_change":"2024-05-09T08:27:55.820639+0000","last_active":"2024-05-09T08:27:57.910235+0000","last_peered":"2024-05-09T08:27:57.910235+0000","last_clean":"2024-05-09T08:27:57.910235+0000","last_became_active":"2024-05-09T08:27:55.820116+0000","last_became_peered":"2024-05-09T08:27:55.820116+0000","last_unstale":"2024-05-09T08:27:57.910235+0000","last_undegraded":"2024-05-09T08:27:57.910235+0000","last_fullsized":"2024-05-09T08:27:57.910235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:35.099440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906516+0000","last_change":"2024-05-09T08:27:55.804807+0000","last_active":"2024-05-09T08:27:57.906516+0000","last_peered":"2024-05-09T08:27:57.906516+0000","last_clean":"2024-05-09T08:27:57.906516+0000","last_became_active":"2024-05-09T08:27:55.804410+0000","last_became_peered":"2024-05-09T08:27:55.804410+0000","last_unstale":"2024-05-09T08:27:57.906516+0000","last_undegraded":"2024-05-09T08:27:57.906516+0000","last_fullsized":"2024-05-09T08:27:57.906516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:03.796554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280679,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":131,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":23048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035709,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1034819,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":443629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":445113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1033995,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:28:05.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:28:05.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 -- 172.21.15.133:0/2803250770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 msgr2=0x7f08501100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 --2- 172.21.15.133:0/2803250770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501100f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0848009ca0 tx=0x7f084802a8b0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 -- 172.21.15.133:0/2803250770 shutdown_connections 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 --2- 172.21.15.133:0/2803250770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501100f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 --2- 172.21.15.133:0/2803250770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850108ea0 0x7f0850109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 --2- 172.21.15.133:0/2803250770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850107cb0 0x7f08501080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 -- 172.21.15.133:0/2803250770 >> 172.21.15.133:0/2803250770 conn(0x7f0850076c70 msgr2=0x7f0850079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 -- 172.21.15.133:0/2803250770 shutdown_connections 2024-05-09T08:28:05.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.349+0000 7f0856e00640 1 -- 172.21.15.133:0/2803250770 wait complete. 2024-05-09T08:28:05.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.350+0000 7f0856e00640 1 Processor -- start 2024-05-09T08:28:05.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.350+0000 7f0856e00640 1 -- start start 2024-05-09T08:28:05.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850107cb0 0x7f085019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:05.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 0x7f085019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0850113980 con 0x7f0850109860 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0850113800 con 0x7f0850108ea0 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0850113b00 con 0x7f0850107cb0 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0855000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 0x7f085019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59572/0 (socket says 172.21.15.133:59572) 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0856400640 1 -- 172.21.15.133:0/4182581846 learned_addr learned my addr 172.21.15.133:0/4182581846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.351+0000 7f0855a00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850107cb0 0x7f085019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 -- 172.21.15.133:0/4182581846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850107cb0 msgr2=0x7f085019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850107cb0 0x7f085019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 -- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 msgr2=0x7f08501a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 -- 172.21.15.133:0/4182581846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08501a5b90 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0856400640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0855000640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 0x7f085019e500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f083c00e9f0 tx=0x7f083c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c00ccc0 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f083c005020 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08501a5e80 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.352+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0049e0 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.353+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08501a62f0 con 0x7f0850108ea0 2024-05-09T08:28:05.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.354+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085010ef70 con 0x7f0850108ea0 2024-05-09T08:28:05.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.356+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f083c005560 con 0x7f0850108ea0 2024-05-09T08:28:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.358+0000 7f084e000640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 0x7f08380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.358+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f083c0cf050 con 0x7f0850108ea0 2024-05-09T08:28:05.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.358+0000 7f0855a00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 0x7f08380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:05.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.358+0000 7f0855a00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 0x7f08380a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0850108d00 tx=0x7f0844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:05.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.362+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f083c0051d0 con 0x7f0850108ea0 2024-05-09T08:28:05.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:05 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3678737078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:05.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:05 smithi133 ceph-mon[29610]: pgmap v320: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-05-09T08:28:05.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.560+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0850061c70 con 0x7f08380a2c20 2024-05-09T08:28:05.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.568+0000 7f084e000640 1 -- 172.21.15.133:0/4182581846 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258948 (secure 0 0 0) 0x7f0850061c70 con 0x7f08380a2c20 2024-05-09T08:28:05.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:05.573 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-05-09T08:28:05.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 msgr2=0x7f08380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 0x7f08380a50e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0850108d00 tx=0x7f0844005f90 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 msgr2=0x7f085019e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 0x7f085019e500 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f083c00e9f0 tx=0x7f083c00eeb0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 shutdown_connections 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.573+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f08380a2c20 0x7f08380a50e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0850109860 0x7f08501a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0850108ea0 0x7f085019e500 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 --2- 172.21.15.133:0/4182581846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0850107cb0 0x7f085019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 >> 172.21.15.133:0/4182581846 conn(0x7f0850076c70 msgr2=0x7f085010c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 shutdown_connections 2024-05-09T08:28:05.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:05.574+0000 7f0856e00640 1 -- 172.21.15.133:0/4182581846 wait complete. 2024-05-09T08:28:05.592 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-05-09T08:28:04.661321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596098,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":152,"num_read_kb":128,"num_write":313,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372828,"kb_used_data":11196,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568996,"statfs":{"total":1151940427776,"available":1151558651904,"internally_reserved":0,"allocated":11464704,"data_stored":7093715,"data_compressed":107103,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5730,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":132,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.882742"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906996+0000","last_change":"2024-05-09T08:27:55.828188+0000","last_active":"2024-05-09T08:27:57.906996+0000","last_peered":"2024-05-09T08:27:57.906996+0000","last_clean":"2024-05-09T08:27:57.906996+0000","last_became_active":"2024-05-09T08:27:55.827858+0000","last_became_peered":"2024-05-09T08:27:55.827858+0000","last_unstale":"2024-05-09T08:27:57.906996+0000","last_undegraded":"2024-05-09T08:27:57.906996+0000","last_fullsized":"2024-05-09T08:27:57.906996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:19.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912235+0000","last_change":"2024-05-09T08:27:55.821504+0000","last_active":"2024-05-09T08:27:57.912235+0000","last_peered":"2024-05-09T08:27:57.912235+0000","last_clean":"2024-05-09T08:27:57.912235+0000","last_became_active":"2024-05-09T08:27:55.818102+0000","last_became_peered":"2024-05-09T08:27:55.818102+0000","last_unstale":"2024-05-09T08:27:57.912235+0000","last_undegraded":"2024-05-09T08:27:57.912235+0000","last_fullsized":"2024-05-09T08:27:57.912235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:00.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911803+0000","last_change":"2024-05-09T08:27:55.817240+0000","last_active":"2024-05-09T08:27:57.911803+0000","last_peered":"2024-05-09T08:27:57.911803+0000","last_clean":"2024-05-09T08:27:57.911803+0000","last_became_active":"2024-05-09T08:27:55.816865+0000","last_became_peered":"2024-05-09T08:27:55.816865+0000","last_unstale":"2024-05-09T08:27:57.911803+0000","last_undegraded":"2024-05-09T08:27:57.911803+0000","last_fullsized":"2024-05-09T08:27:57.911803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:04.149531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.781033+0000","last_change":"2024-05-09T08:27:55.805704+0000","last_active":"2024-05-09T08:27:56.781033+0000","last_peered":"2024-05-09T08:27:56.781033+0000","last_clean":"2024-05-09T08:27:56.781033+0000","last_became_active":"2024-05-09T08:27:55.805344+0000","last_became_peered":"2024-05-09T08:27:55.805344+0000","last_unstale":"2024-05-09T08:27:56.781033+0000","last_undegraded":"2024-05-09T08:27:56.781033+0000","last_fullsized":"2024-05-09T08:27:56.781033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:02.418060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224767+0000","last_change":"2024-05-09T08:27:55.821747+0000","last_active":"2024-05-09T08:27:57.224767+0000","last_peered":"2024-05-09T08:27:57.224767+0000","last_clean":"2024-05-09T08:27:57.224767+0000","last_became_active":"2024-05-09T08:27:55.821378+0000","last_became_peered":"2024-05-09T08:27:55.821378+0000","last_unstale":"2024-05-09T08:27:57.224767+0000","last_undegraded":"2024-05-09T08:27:57.224767+0000","last_fullsized":"2024-05-09T08:27:57.224767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:53.470723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978180+0000","last_change":"2024-05-09T08:27:55.812827+0000","last_active":"2024-05-09T08:27:56.978180+0000","last_peered":"2024-05-09T08:27:56.978180+0000","last_clean":"2024-05-09T08:27:56.978180+0000","last_became_active":"2024-05-09T08:27:55.812412+0000","last_became_peered":"2024-05-09T08:27:55.812412+0000","last_unstale":"2024-05-09T08:27:56.978180+0000","last_undegraded":"2024-05-09T08:27:56.978180+0000","last_fullsized":"2024-05-09T08:27:56.978180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:20.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387427+0000","last_change":"2024-05-09T08:27:55.811536+0000","last_active":"2024-05-09T08:27:57.387427+0000","last_peered":"2024-05-09T08:27:57.387427+0000","last_clean":"2024-05-09T08:27:57.387427+0000","last_became_active":"2024-05-09T08:27:55.811181+0000","last_became_peered":"2024-05-09T08:27:55.811181+0000","last_unstale":"2024-05-09T08:27:57.387427+0000","last_undegraded":"2024-05-09T08:27:57.387427+0000","last_fullsized":"2024-05-09T08:27:57.387427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227101+0000","last_change":"2024-05-09T08:27:55.823372+0000","last_active":"2024-05-09T08:27:57.227101+0000","last_peered":"2024-05-09T08:27:57.227101+0000","last_clean":"2024-05-09T08:27:57.227101+0000","last_became_active":"2024-05-09T08:27:55.823052+0000","last_became_peered":"2024-05-09T08:27:55.823052+0000","last_unstale":"2024-05-09T08:27:57.227101+0000","last_undegraded":"2024-05-09T08:27:57.227101+0000","last_fullsized":"2024-05-09T08:27:57.227101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:53:18.677587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906989+0000","last_change":"2024-05-09T08:27:55.802154+0000","last_active":"2024-05-09T08:27:57.906989+0000","last_peered":"2024-05-09T08:27:57.906989+0000","last_clean":"2024-05-09T08:27:57.906989+0000","last_became_active":"2024-05-09T08:27:55.801501+0000","last_became_peered":"2024-05-09T08:27:55.801501+0000","last_unstale":"2024-05-09T08:27:57.906989+0000","last_undegraded":"2024-05-09T08:27:57.906989+0000","last_fullsized":"2024-05-09T08:27:57.906989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:42.009521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978464+0000","last_change":"2024-05-09T08:27:55.810294+0000","last_active":"2024-05-09T08:27:56.978464+0000","last_peered":"2024-05-09T08:27:56.978464+0000","last_clean":"2024-05-09T08:27:56.978464+0000","last_became_active":"2024-05-09T08:27:55.809825+0000","last_became_peered":"2024-05-09T08:27:55.809825+0000","last_unstale":"2024-05-09T08:27:56.978464+0000","last_undegraded":"2024-05-09T08:27:56.978464+0000","last_fullsized":"2024-05-09T08:27:56.978464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:39:39.846941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387653+0000","last_change":"2024-05-09T08:27:55.806122+0000","last_active":"2024-05-09T08:27:57.387653+0000","last_peered":"2024-05-09T08:27:57.387653+0000","last_clean":"2024-05-09T08:27:57.387653+0000","last_became_active":"2024-05-09T08:27:55.803844+0000","last_became_peered":"2024-05-09T08:27:55.803844+0000","last_unstale":"2024-05-09T08:27:57.387653+0000","last_undegraded":"2024-05-09T08:27:57.387653+0000","last_fullsized":"2024-05-09T08:27:57.387653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:40.692633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780958+0000","last_change":"2024-05-09T08:27:55.810184+0000","last_active":"2024-05-09T08:27:56.780958+0000","last_peered":"2024-05-09T08:27:56.780958+0000","last_clean":"2024-05-09T08:27:56.780958+0000","last_became_active":"2024-05-09T08:27:55.809833+0000","last_became_peered":"2024-05-09T08:27:55.809833+0000","last_unstale":"2024-05-09T08:27:56.780958+0000","last_undegraded":"2024-05-09T08:27:56.780958+0000","last_fullsized":"2024-05-09T08:27:56.780958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:19:33.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909787+0000","last_change":"2024-05-09T08:27:55.804770+0000","last_active":"2024-05-09T08:27:57.909787+0000","last_peered":"2024-05-09T08:27:57.909787+0000","last_clean":"2024-05-09T08:27:57.909787+0000","last_became_active":"2024-05-09T08:27:55.803486+0000","last_became_peered":"2024-05-09T08:27:55.803486+0000","last_unstale":"2024-05-09T08:27:57.909787+0000","last_undegraded":"2024-05-09T08:27:57.909787+0000","last_fullsized":"2024-05-09T08:27:57.909787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:28.641874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910780+0000","last_change":"2024-05-09T08:27:55.826032+0000","last_active":"2024-05-09T08:27:57.910780+0000","last_peered":"2024-05-09T08:27:57.910780+0000","last_clean":"2024-05-09T08:27:57.910780+0000","last_became_active":"2024-05-09T08:27:55.825510+0000","last_became_peered":"2024-05-09T08:27:55.825510+0000","last_unstale":"2024-05-09T08:27:57.910780+0000","last_undegraded":"2024-05-09T08:27:57.910780+0000","last_fullsized":"2024-05-09T08:27:57.910780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:25.691361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911177+0000","last_change":"2024-05-09T08:27:55.816178+0000","last_active":"2024-05-09T08:27:57.911177+0000","last_peered":"2024-05-09T08:27:57.911177+0000","last_clean":"2024-05-09T08:27:57.911177+0000","last_became_active":"2024-05-09T08:27:55.814592+0000","last_became_peered":"2024-05-09T08:27:55.814592+0000","last_unstale":"2024-05-09T08:27:57.911177+0000","last_undegraded":"2024-05-09T08:27:57.911177+0000","last_fullsized":"2024-05-09T08:27:57.911177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:06.632314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906451+0000","last_change":"2024-05-09T08:27:55.827914+0000","last_active":"2024-05-09T08:27:57.906451+0000","last_peered":"2024-05-09T08:27:57.906451+0000","last_clean":"2024-05-09T08:27:57.906451+0000","last_became_active":"2024-05-09T08:27:55.827555+0000","last_became_peered":"2024-05-09T08:27:55.827555+0000","last_unstale":"2024-05-09T08:27:57.906451+0000","last_undegraded":"2024-05-09T08:27:57.906451+0000","last_fullsized":"2024-05-09T08:27:57.906451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:37.097461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907181+0000","last_change":"2024-05-09T08:27:55.828072+0000","last_active":"2024-05-09T08:27:57.907181+0000","last_peered":"2024-05-09T08:27:57.907181+0000","last_clean":"2024-05-09T08:27:57.907181+0000","last_became_active":"2024-05-09T08:27:55.827727+0000","last_became_peered":"2024-05-09T08:27:55.827727+0000","last_unstale":"2024-05-09T08:27:57.907181+0000","last_undegraded":"2024-05-09T08:27:57.907181+0000","last_fullsized":"2024-05-09T08:27:57.907181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:13.761560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.782409+0000","last_change":"2024-05-09T08:27:55.812506+0000","last_active":"2024-05-09T08:27:56.782409+0000","last_peered":"2024-05-09T08:27:56.782409+0000","last_clean":"2024-05-09T08:27:56.782409+0000","last_became_active":"2024-05-09T08:27:55.812071+0000","last_became_peered":"2024-05-09T08:27:55.812071+0000","last_unstale":"2024-05-09T08:27:56.782409+0000","last_undegraded":"2024-05-09T08:27:56.782409+0000","last_fullsized":"2024-05-09T08:27:56.782409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:53.318140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780716+0000","last_change":"2024-05-09T08:27:55.817660+0000","last_active":"2024-05-09T08:27:56.780716+0000","last_peered":"2024-05-09T08:27:56.780716+0000","last_clean":"2024-05-09T08:27:56.780716+0000","last_became_active":"2024-05-09T08:27:55.817357+0000","last_became_peered":"2024-05-09T08:27:55.817357+0000","last_unstale":"2024-05-09T08:27:56.780716+0000","last_undegraded":"2024-05-09T08:27:56.780716+0000","last_fullsized":"2024-05-09T08:27:56.780716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:05.489063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270532+0000","last_change":"2024-05-09T08:27:54.784177+0000","last_active":"2024-05-09T08:28:00.270532+0000","last_peered":"2024-05-09T08:28:00.270532+0000","last_clean":"2024-05-09T08:28:00.270532+0000","last_became_active":"2024-05-09T08:27:54.780982+0000","last_became_peered":"2024-05-09T08:27:54.780982+0000","last_unstale":"2024-05-09T08:28:00.270532+0000","last_undegraded":"2024-05-09T08:28:00.270532+0000","last_fullsized":"2024-05-09T08:28:00.270532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:17.668694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226734+0000","last_change":"2024-05-09T08:27:55.825495+0000","last_active":"2024-05-09T08:27:57.226734+0000","last_peered":"2024-05-09T08:27:57.226734+0000","last_clean":"2024-05-09T08:27:57.226734+0000","last_became_active":"2024-05-09T08:27:55.821680+0000","last_became_peered":"2024-05-09T08:27:55.821680+0000","last_unstale":"2024-05-09T08:27:57.226734+0000","last_undegraded":"2024-05-09T08:27:57.226734+0000","last_fullsized":"2024-05-09T08:27:57.226734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:06.128200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:58.704233+0000","last_change":"2024-05-09T08:27:54.791362+0000","last_active":"2024-05-09T08:27:58.704233+0000","last_peered":"2024-05-09T08:27:58.704233+0000","last_clean":"2024-05-09T08:27:58.704233+0000","last_became_active":"2024-05-09T08:27:54.791050+0000","last_became_peered":"2024-05-09T08:27:54.791050+0000","last_unstale":"2024-05-09T08:27:58.704233+0000","last_undegraded":"2024-05-09T08:27:58.704233+0000","last_fullsized":"2024-05-09T08:27:58.704233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:11.821907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912541+0000","last_change":"2024-05-09T08:27:55.820278+0000","last_active":"2024-05-09T08:27:57.912541+0000","last_peered":"2024-05-09T08:27:57.912541+0000","last_clean":"2024-05-09T08:27:57.912541+0000","last_became_active":"2024-05-09T08:27:55.819861+0000","last_became_peered":"2024-05-09T08:27:55.819861+0000","last_unstale":"2024-05-09T08:27:57.912541+0000","last_undegraded":"2024-05-09T08:27:57.912541+0000","last_fullsized":"2024-05-09T08:27:57.912541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:29.541259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.267857+0000","last_change":"2024-05-09T08:27:54.771453+0000","last_active":"2024-05-09T08:28:00.267857+0000","last_peered":"2024-05-09T08:28:00.267857+0000","last_clean":"2024-05-09T08:28:00.267857+0000","last_became_active":"2024-05-09T08:27:54.770746+0000","last_became_peered":"2024-05-09T08:27:54.770746+0000","last_unstale":"2024-05-09T08:28:00.267857+0000","last_undegraded":"2024-05-09T08:28:00.267857+0000","last_fullsized":"2024-05-09T08:28:00.267857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:18.823562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906537+0000","last_change":"2024-05-09T08:27:55.803341+0000","last_active":"2024-05-09T08:27:57.906537+0000","last_peered":"2024-05-09T08:27:57.906537+0000","last_clean":"2024-05-09T08:27:57.906537+0000","last_became_active":"2024-05-09T08:27:55.802945+0000","last_became_peered":"2024-05-09T08:27:55.802945+0000","last_unstale":"2024-05-09T08:27:57.906537+0000","last_undegraded":"2024-05-09T08:27:57.906537+0000","last_fullsized":"2024-05-09T08:27:57.906537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:24.346139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225059+0000","last_change":"2024-05-09T08:27:54.769140+0000","last_active":"2024-05-09T08:27:57.225059+0000","last_peered":"2024-05-09T08:27:57.225059+0000","last_clean":"2024-05-09T08:27:57.225059+0000","last_became_active":"2024-05-09T08:27:54.768615+0000","last_became_peered":"2024-05-09T08:27:54.768615+0000","last_unstale":"2024-05-09T08:27:57.225059+0000","last_undegraded":"2024-05-09T08:27:57.225059+0000","last_fullsized":"2024-05-09T08:27:57.225059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:10.016070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907255+0000","last_change":"2024-05-09T08:27:55.806385+0000","last_active":"2024-05-09T08:27:57.907255+0000","last_peered":"2024-05-09T08:27:57.907255+0000","last_clean":"2024-05-09T08:27:57.907255+0000","last_became_active":"2024-05-09T08:27:55.806031+0000","last_became_peered":"2024-05-09T08:27:55.806031+0000","last_unstale":"2024-05-09T08:27:57.907255+0000","last_undegraded":"2024-05-09T08:27:57.907255+0000","last_fullsized":"2024-05-09T08:27:57.907255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:17.326277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.282887+0000","last_change":"2024-05-09T08:27:54.769148+0000","last_active":"2024-05-09T08:28:02.282887+0000","last_peered":"2024-05-09T08:28:02.282887+0000","last_clean":"2024-05-09T08:28:02.282887+0000","last_became_active":"2024-05-09T08:27:54.768814+0000","last_became_peered":"2024-05-09T08:27:54.768814+0000","last_unstale":"2024-05-09T08:28:02.282887+0000","last_undegraded":"2024-05-09T08:28:02.282887+0000","last_fullsized":"2024-05-09T08:28:02.282887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:08.276294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224830+0000","last_change":"2024-05-09T08:27:55.812884+0000","last_active":"2024-05-09T08:27:57.224830+0000","last_peered":"2024-05-09T08:27:57.224830+0000","last_clean":"2024-05-09T08:27:57.224830+0000","last_became_active":"2024-05-09T08:27:55.812515+0000","last_became_peered":"2024-05-09T08:27:55.812515+0000","last_unstale":"2024-05-09T08:27:57.224830+0000","last_undegraded":"2024-05-09T08:27:57.224830+0000","last_fullsized":"2024-05-09T08:27:57.224830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:59.211293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.268419+0000","last_change":"2024-05-09T08:27:54.764806+0000","last_active":"2024-05-09T08:28:00.268419+0000","last_peered":"2024-05-09T08:28:00.268419+0000","last_clean":"2024-05-09T08:28:00.268419+0000","last_became_active":"2024-05-09T08:27:54.764325+0000","last_became_peered":"2024-05-09T08:27:54.764325+0000","last_unstale":"2024-05-09T08:28:00.268419+0000","last_undegraded":"2024-05-09T08:28:00.268419+0000","last_fullsized":"2024-05-09T08:28:00.268419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:11.820240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907385+0000","last_change":"2024-05-09T08:27:55.822481+0000","last_active":"2024-05-09T08:27:57.907385+0000","last_peered":"2024-05-09T08:27:57.907385+0000","last_clean":"2024-05-09T08:27:57.907385+0000","last_became_active":"2024-05-09T08:27:55.821180+0000","last_became_peered":"2024-05-09T08:27:55.821180+0000","last_unstale":"2024-05-09T08:27:57.907385+0000","last_undegraded":"2024-05-09T08:27:57.907385+0000","last_fullsized":"2024-05-09T08:27:57.907385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:33:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.271850+0000","last_change":"2024-05-09T08:27:54.771263+0000","last_active":"2024-05-09T08:28:00.271850+0000","last_peered":"2024-05-09T08:28:00.271850+0000","last_clean":"2024-05-09T08:28:00.271850+0000","last_became_active":"2024-05-09T08:27:54.770719+0000","last_became_peered":"2024-05-09T08:27:54.770719+0000","last_unstale":"2024-05-09T08:28:00.271850+0000","last_undegraded":"2024-05-09T08:28:00.271850+0000","last_fullsized":"2024-05-09T08:28:00.271850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:08.725081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226429+0000","last_change":"2024-05-09T08:27:55.821493+0000","last_active":"2024-05-09T08:27:57.226429+0000","last_peered":"2024-05-09T08:27:57.226429+0000","last_clean":"2024-05-09T08:27:57.226429+0000","last_became_active":"2024-05-09T08:27:55.818585+0000","last_became_peered":"2024-05-09T08:27:55.818585+0000","last_unstale":"2024-05-09T08:27:57.226429+0000","last_undegraded":"2024-05-09T08:27:57.226429+0000","last_fullsized":"2024-05-09T08:27:57.226429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:56.319691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906205+0000","last_change":"2024-05-09T08:27:54.769856+0000","last_active":"2024-05-09T08:27:57.906205+0000","last_peered":"2024-05-09T08:27:57.906205+0000","last_clean":"2024-05-09T08:27:57.906205+0000","last_became_active":"2024-05-09T08:27:54.769548+0000","last_became_peered":"2024-05-09T08:27:54.769548+0000","last_unstale":"2024-05-09T08:27:57.906205+0000","last_undegraded":"2024-05-09T08:27:57.906205+0000","last_fullsized":"2024-05-09T08:27:57.906205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:57.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.781667+0000","last_change":"2024-05-09T08:27:55.822063+0000","last_active":"2024-05-09T08:27:56.781667+0000","last_peered":"2024-05-09T08:27:56.781667+0000","last_clean":"2024-05-09T08:27:56.781667+0000","last_became_active":"2024-05-09T08:27:55.821624+0000","last_became_peered":"2024-05-09T08:27:55.821624+0000","last_unstale":"2024-05-09T08:27:56.781667+0000","last_undegraded":"2024-05-09T08:27:56.781667+0000","last_fullsized":"2024-05-09T08:27:56.781667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:14.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.259800+0000","last_change":"2024-05-09T08:27:54.767639+0000","last_active":"2024-05-09T08:27:59.259800+0000","last_peered":"2024-05-09T08:27:59.259800+0000","last_clean":"2024-05-09T08:27:59.259800+0000","last_became_active":"2024-05-09T08:27:54.767284+0000","last_became_peered":"2024-05-09T08:27:54.767284+0000","last_unstale":"2024-05-09T08:27:59.259800+0000","last_undegraded":"2024-05-09T08:27:59.259800+0000","last_fullsized":"2024-05-09T08:27:59.259800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:07.110319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906933+0000","last_change":"2024-05-09T08:27:55.817050+0000","last_active":"2024-05-09T08:27:57.906933+0000","last_peered":"2024-05-09T08:27:57.906933+0000","last_clean":"2024-05-09T08:27:57.906933+0000","last_became_active":"2024-05-09T08:27:55.816655+0000","last_became_peered":"2024-05-09T08:27:55.816655+0000","last_unstale":"2024-05-09T08:27:57.906933+0000","last_undegraded":"2024-05-09T08:27:57.906933+0000","last_fullsized":"2024-05-09T08:27:57.906933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:06.132482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258793+0000","last_change":"2024-05-09T08:27:54.770449+0000","last_active":"2024-05-09T08:27:59.258793+0000","last_peered":"2024-05-09T08:27:59.258793+0000","last_clean":"2024-05-09T08:27:59.258793+0000","last_became_active":"2024-05-09T08:27:54.769382+0000","last_became_peered":"2024-05-09T08:27:54.769382+0000","last_unstale":"2024-05-09T08:27:59.258793+0000","last_undegraded":"2024-05-09T08:27:59.258793+0000","last_fullsized":"2024-05-09T08:27:59.258793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:47.108749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912376+0000","last_change":"2024-05-09T08:27:55.831416+0000","last_active":"2024-05-09T08:27:57.912376+0000","last_peered":"2024-05-09T08:27:57.912376+0000","last_clean":"2024-05-09T08:27:57.912376+0000","last_became_active":"2024-05-09T08:27:55.831011+0000","last_became_peered":"2024-05-09T08:27:55.831011+0000","last_unstale":"2024-05-09T08:27:57.912376+0000","last_undegraded":"2024-05-09T08:27:57.912376+0000","last_fullsized":"2024-05-09T08:27:57.912376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:23.820425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258788+0000","last_change":"2024-05-09T08:27:54.776737+0000","last_active":"2024-05-09T08:27:59.258788+0000","last_peered":"2024-05-09T08:27:59.258788+0000","last_clean":"2024-05-09T08:27:59.258788+0000","last_became_active":"2024-05-09T08:27:54.776402+0000","last_became_peered":"2024-05-09T08:27:54.776402+0000","last_unstale":"2024-05-09T08:27:59.258788+0000","last_undegraded":"2024-05-09T08:27:59.258788+0000","last_fullsized":"2024-05-09T08:27:59.258788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:38.135159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906693+0000","last_change":"2024-05-09T08:27:55.810092+0000","last_active":"2024-05-09T08:27:57.906693+0000","last_peered":"2024-05-09T08:27:57.906693+0000","last_clean":"2024-05-09T08:27:57.906693+0000","last_became_active":"2024-05-09T08:27:55.809738+0000","last_became_peered":"2024-05-09T08:27:55.809738+0000","last_unstale":"2024-05-09T08:27:57.906693+0000","last_undegraded":"2024-05-09T08:27:57.906693+0000","last_fullsized":"2024-05-09T08:27:57.906693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:34.094014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:01.282643+0000","last_change":"2024-05-09T08:27:54.769608+0000","last_active":"2024-05-09T08:28:01.282643+0000","last_peered":"2024-05-09T08:28:01.282643+0000","last_clean":"2024-05-09T08:28:01.282643+0000","last_became_active":"2024-05-09T08:27:54.769286+0000","last_became_peered":"2024-05-09T08:27:54.769286+0000","last_unstale":"2024-05-09T08:28:01.282643+0000","last_undegraded":"2024-05-09T08:28:01.282643+0000","last_fullsized":"2024-05-09T08:28:01.282643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:40.620602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225059+0000","last_change":"2024-05-09T08:27:55.800489+0000","last_active":"2024-05-09T08:27:57.225059+0000","last_peered":"2024-05-09T08:27:57.225059+0000","last_clean":"2024-05-09T08:27:57.225059+0000","last_became_active":"2024-05-09T08:27:55.800145+0000","last_became_peered":"2024-05-09T08:27:55.800145+0000","last_unstale":"2024-05-09T08:27:57.225059+0000","last_undegraded":"2024-05-09T08:27:57.225059+0000","last_fullsized":"2024-05-09T08:27:57.225059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:47.877630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.271696+0000","last_change":"2024-05-09T08:27:54.779179+0000","last_active":"2024-05-09T08:28:00.271696+0000","last_peered":"2024-05-09T08:28:00.271696+0000","last_clean":"2024-05-09T08:28:00.271696+0000","last_became_active":"2024-05-09T08:27:54.778793+0000","last_became_peered":"2024-05-09T08:27:54.778793+0000","last_unstale":"2024-05-09T08:28:00.271696+0000","last_undegraded":"2024-05-09T08:28:00.271696+0000","last_fullsized":"2024-05-09T08:28:00.271696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:56:42.414959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909073+0000","last_change":"2024-05-09T08:27:55.830500+0000","last_active":"2024-05-09T08:27:57.909073+0000","last_peered":"2024-05-09T08:27:57.909073+0000","last_clean":"2024-05-09T08:27:57.909073+0000","last_became_active":"2024-05-09T08:27:55.830044+0000","last_became_peered":"2024-05-09T08:27:55.830044+0000","last_unstale":"2024-05-09T08:27:57.909073+0000","last_undegraded":"2024-05-09T08:27:57.909073+0000","last_fullsized":"2024-05-09T08:27:57.909073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:15.616303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.260714+0000","last_change":"2024-05-09T08:27:54.786607+0000","last_active":"2024-05-09T08:27:59.260714+0000","last_peered":"2024-05-09T08:27:59.260714+0000","last_clean":"2024-05-09T08:27:59.260714+0000","last_became_active":"2024-05-09T08:27:54.786286+0000","last_became_peered":"2024-05-09T08:27:54.786286+0000","last_unstale":"2024-05-09T08:27:59.260714+0000","last_undegraded":"2024-05-09T08:27:59.260714+0000","last_fullsized":"2024-05-09T08:27:59.260714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.833237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387752+0000","last_change":"2024-05-09T08:27:55.809022+0000","last_active":"2024-05-09T08:27:57.387752+0000","last_peered":"2024-05-09T08:27:57.387752+0000","last_clean":"2024-05-09T08:27:57.387752+0000","last_became_active":"2024-05-09T08:27:55.808666+0000","last_became_peered":"2024-05-09T08:27:55.808666+0000","last_unstale":"2024-05-09T08:27:57.387752+0000","last_undegraded":"2024-05-09T08:27:57.387752+0000","last_fullsized":"2024-05-09T08:27:57.387752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:18.339576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.287215+0000","last_change":"2024-05-09T08:27:54.769166+0000","last_active":"2024-05-09T08:28:02.287215+0000","last_peered":"2024-05-09T08:28:02.287215+0000","last_clean":"2024-05-09T08:28:02.287215+0000","last_became_active":"2024-05-09T08:27:54.768794+0000","last_became_peered":"2024-05-09T08:27:54.768794+0000","last_unstale":"2024-05-09T08:28:02.287215+0000","last_undegraded":"2024-05-09T08:28:02.287215+0000","last_fullsized":"2024-05-09T08:28:02.287215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.222388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387305+0000","last_change":"2024-05-09T08:27:55.806685+0000","last_active":"2024-05-09T08:27:57.387305+0000","last_peered":"2024-05-09T08:27:57.387305+0000","last_clean":"2024-05-09T08:27:57.387305+0000","last_became_active":"2024-05-09T08:27:55.804641+0000","last_became_peered":"2024-05-09T08:27:55.804641+0000","last_unstale":"2024-05-09T08:27:57.387305+0000","last_undegraded":"2024-05-09T08:27:57.387305+0000","last_fullsized":"2024-05-09T08:27:57.387305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:40.307764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270776+0000","last_change":"2024-05-09T08:27:54.782524+0000","last_active":"2024-05-09T08:28:00.270776+0000","last_peered":"2024-05-09T08:28:00.270776+0000","last_clean":"2024-05-09T08:28:00.270776+0000","last_became_active":"2024-05-09T08:27:54.782220+0000","last_became_peered":"2024-05-09T08:27:54.782220+0000","last_unstale":"2024-05-09T08:28:00.270776+0000","last_undegraded":"2024-05-09T08:28:00.270776+0000","last_fullsized":"2024-05-09T08:28:00.270776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:16:58.705682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906487+0000","last_change":"2024-05-09T08:27:55.806390+0000","last_active":"2024-05-09T08:27:57.906487+0000","last_peered":"2024-05-09T08:27:57.906487+0000","last_clean":"2024-05-09T08:27:57.906487+0000","last_became_active":"2024-05-09T08:27:55.806027+0000","last_became_peered":"2024-05-09T08:27:55.806027+0000","last_unstale":"2024-05-09T08:27:57.906487+0000","last_undegraded":"2024-05-09T08:27:57.906487+0000","last_fullsized":"2024-05-09T08:27:57.906487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:58.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225231+0000","last_change":"2024-05-09T08:27:54.766679+0000","last_active":"2024-05-09T08:27:57.225231+0000","last_peered":"2024-05-09T08:27:57.225231+0000","last_clean":"2024-05-09T08:27:57.225231+0000","last_became_active":"2024-05-09T08:27:54.766318+0000","last_became_peered":"2024-05-09T08:27:54.766318+0000","last_unstale":"2024-05-09T08:27:57.225231+0000","last_undegraded":"2024-05-09T08:27:57.225231+0000","last_fullsized":"2024-05-09T08:27:57.225231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:38.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906734+0000","last_change":"2024-05-09T08:27:55.804379+0000","last_active":"2024-05-09T08:27:57.906734+0000","last_peered":"2024-05-09T08:27:57.906734+0000","last_clean":"2024-05-09T08:27:57.906734+0000","last_became_active":"2024-05-09T08:27:55.804046+0000","last_became_peered":"2024-05-09T08:27:55.804046+0000","last_unstale":"2024-05-09T08:27:57.906734+0000","last_undegraded":"2024-05-09T08:27:57.906734+0000","last_fullsized":"2024-05-09T08:27:57.906734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:43.735487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.782028+0000","last_change":"2024-05-09T08:27:54.785336+0000","last_active":"2024-05-09T08:27:56.782028+0000","last_peered":"2024-05-09T08:27:56.782028+0000","last_clean":"2024-05-09T08:27:56.782028+0000","last_became_active":"2024-05-09T08:27:54.784581+0000","last_became_peered":"2024-05-09T08:27:54.784581+0000","last_unstale":"2024-05-09T08:27:56.782028+0000","last_undegraded":"2024-05-09T08:27:56.782028+0000","last_fullsized":"2024-05-09T08:27:56.782028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:40.071747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978840+0000","last_change":"2024-05-09T08:27:55.821512+0000","last_active":"2024-05-09T08:27:56.978840+0000","last_peered":"2024-05-09T08:27:56.978840+0000","last_clean":"2024-05-09T08:27:56.978840+0000","last_became_active":"2024-05-09T08:27:55.821162+0000","last_became_peered":"2024-05-09T08:27:55.821162+0000","last_unstale":"2024-05-09T08:27:56.978840+0000","last_undegraded":"2024-05-09T08:27:56.978840+0000","last_fullsized":"2024-05-09T08:27:56.978840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:47.377452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387417+0000","last_change":"2024-05-09T08:27:54.771528+0000","last_active":"2024-05-09T08:27:57.387417+0000","last_peered":"2024-05-09T08:27:57.387417+0000","last_clean":"2024-05-09T08:27:57.387417+0000","last_became_active":"2024-05-09T08:27:54.771133+0000","last_became_peered":"2024-05-09T08:27:54.771133+0000","last_unstale":"2024-05-09T08:27:57.387417+0000","last_undegraded":"2024-05-09T08:27:57.387417+0000","last_fullsized":"2024-05-09T08:27:57.387417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:22.948800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.978312+0000","last_change":"2024-05-09T08:27:55.824954+0000","last_active":"2024-05-09T08:27:56.978312+0000","last_peered":"2024-05-09T08:27:56.978312+0000","last_clean":"2024-05-09T08:27:56.978312+0000","last_became_active":"2024-05-09T08:27:55.824578+0000","last_became_peered":"2024-05-09T08:27:55.824578+0000","last_unstale":"2024-05-09T08:27:56.978312+0000","last_undegraded":"2024-05-09T08:27:56.978312+0000","last_fullsized":"2024-05-09T08:27:56.978312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:35:22.790193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906602+0000","last_change":"2024-05-09T08:27:55.804508+0000","last_active":"2024-05-09T08:27:57.906602+0000","last_peered":"2024-05-09T08:27:57.906602+0000","last_clean":"2024-05-09T08:27:57.906602+0000","last_became_active":"2024-05-09T08:27:55.804190+0000","last_became_peered":"2024-05-09T08:27:55.804190+0000","last_unstale":"2024-05-09T08:27:57.906602+0000","last_undegraded":"2024-05-09T08:27:57.906602+0000","last_fullsized":"2024-05-09T08:27:57.906602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:24.232140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:02.285504+0000","last_change":"2024-05-09T08:27:54.780486+0000","last_active":"2024-05-09T08:28:02.285504+0000","last_peered":"2024-05-09T08:28:02.285504+0000","last_clean":"2024-05-09T08:28:02.285504+0000","last_became_active":"2024-05-09T08:27:54.777349+0000","last_became_peered":"2024-05-09T08:27:54.777349+0000","last_unstale":"2024-05-09T08:28:02.285504+0000","last_undegraded":"2024-05-09T08:28:02.285504+0000","last_fullsized":"2024-05-09T08:28:02.285504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:14.729741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226851+0000","last_change":"2024-05-09T08:27:55.825469+0000","last_active":"2024-05-09T08:27:57.226851+0000","last_peered":"2024-05-09T08:27:57.226851+0000","last_clean":"2024-05-09T08:27:57.226851+0000","last_became_active":"2024-05-09T08:27:55.825103+0000","last_became_peered":"2024-05-09T08:27:55.825103+0000","last_unstale":"2024-05-09T08:27:57.226851+0000","last_undegraded":"2024-05-09T08:27:57.226851+0000","last_fullsized":"2024-05-09T08:27:57.226851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:53.136741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:01.278178+0000","last_change":"2024-05-09T08:27:54.774194+0000","last_active":"2024-05-09T08:28:01.278178+0000","last_peered":"2024-05-09T08:28:01.278178+0000","last_clean":"2024-05-09T08:28:01.278178+0000","last_became_active":"2024-05-09T08:27:54.773678+0000","last_became_peered":"2024-05-09T08:27:54.773678+0000","last_unstale":"2024-05-09T08:28:01.278178+0000","last_undegraded":"2024-05-09T08:28:01.278178+0000","last_fullsized":"2024-05-09T08:28:01.278178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:23.199301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387185+0000","last_change":"2024-05-09T08:26:34.966221+0000","last_active":"2024-05-09T08:27:57.387185+0000","last_peered":"2024-05-09T08:27:57.387185+0000","last_clean":"2024-05-09T08:27:57.387185+0000","last_became_active":"2024-05-09T08:26:34.965800+0000","last_became_peered":"2024-05-09T08:26:34.965800+0000","last_unstale":"2024-05-09T08:27:57.387185+0000","last_undegraded":"2024-05-09T08:27:57.387185+0000","last_fullsized":"2024-05-09T08:27:57.387185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_clean_scrub_stamp":"2024-05-09T08:22:41.596226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:10.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909184+0000","last_change":"2024-05-09T08:27:55.805417+0000","last_active":"2024-05-09T08:27:57.909184+0000","last_peered":"2024-05-09T08:27:57.909184+0000","last_clean":"2024-05-09T08:27:57.909184+0000","last_became_active":"2024-05-09T08:27:55.805025+0000","last_became_peered":"2024-05-09T08:27:55.805025+0000","last_unstale":"2024-05-09T08:27:57.909184+0000","last_undegraded":"2024-05-09T08:27:57.909184+0000","last_fullsized":"2024-05-09T08:27:57.909184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:35.442848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.267533+0000","last_change":"2024-05-09T08:27:54.769276+0000","last_active":"2024-05-09T08:28:00.267533+0000","last_peered":"2024-05-09T08:28:00.267533+0000","last_clean":"2024-05-09T08:28:00.267533+0000","last_became_active":"2024-05-09T08:27:54.768650+0000","last_became_peered":"2024-05-09T08:27:54.768650+0000","last_unstale":"2024-05-09T08:28:00.267533+0000","last_undegraded":"2024-05-09T08:28:00.267533+0000","last_fullsized":"2024-05-09T08:28:00.267533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:32.480610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.225151+0000","last_change":"2024-05-09T08:27:54.782809+0000","last_active":"2024-05-09T08:27:57.225151+0000","last_peered":"2024-05-09T08:27:57.225151+0000","last_clean":"2024-05-09T08:27:57.225151+0000","last_became_active":"2024-05-09T08:27:54.782452+0000","last_became_peered":"2024-05-09T08:27:54.782452+0000","last_unstale":"2024-05-09T08:27:57.225151+0000","last_undegraded":"2024-05-09T08:27:57.225151+0000","last_fullsized":"2024-05-09T08:27:57.225151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:42.371573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909128+0000","last_change":"2024-05-09T08:27:55.814683+0000","last_active":"2024-05-09T08:27:57.909128+0000","last_peered":"2024-05-09T08:27:57.909128+0000","last_clean":"2024-05-09T08:27:57.909128+0000","last_became_active":"2024-05-09T08:27:55.813788+0000","last_became_peered":"2024-05-09T08:27:55.813788+0000","last_unstale":"2024-05-09T08:27:57.909128+0000","last_undegraded":"2024-05-09T08:27:57.909128+0000","last_fullsized":"2024-05-09T08:27:57.909128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:45.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:03.225474+0000","last_change":"2024-05-09T08:27:54.788393+0000","last_active":"2024-05-09T08:28:03.225474+0000","last_peered":"2024-05-09T08:28:03.225474+0000","last_clean":"2024-05-09T08:28:03.225474+0000","last_became_active":"2024-05-09T08:27:54.788031+0000","last_became_peered":"2024-05-09T08:27:54.788031+0000","last_unstale":"2024-05-09T08:28:03.225474+0000","last_undegraded":"2024-05-09T08:28:03.225474+0000","last_fullsized":"2024-05-09T08:28:03.225474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:26.451385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227322+0000","last_change":"2024-05-09T08:27:55.812391+0000","last_active":"2024-05-09T08:27:57.227322+0000","last_peered":"2024-05-09T08:27:57.227322+0000","last_clean":"2024-05-09T08:27:57.227322+0000","last_became_active":"2024-05-09T08:27:55.811954+0000","last_became_peered":"2024-05-09T08:27:55.811954+0000","last_unstale":"2024-05-09T08:27:57.227322+0000","last_undegraded":"2024-05-09T08:27:57.227322+0000","last_fullsized":"2024-05-09T08:27:57.227322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:38.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258030+0000","last_change":"2024-05-09T08:27:54.775693+0000","last_active":"2024-05-09T08:27:59.258030+0000","last_peered":"2024-05-09T08:27:59.258030+0000","last_clean":"2024-05-09T08:27:59.258030+0000","last_became_active":"2024-05-09T08:27:54.775198+0000","last_became_peered":"2024-05-09T08:27:54.775198+0000","last_unstale":"2024-05-09T08:27:59.258030+0000","last_undegraded":"2024-05-09T08:27:59.258030+0000","last_fullsized":"2024-05-09T08:27:59.258030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:21:18.672744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.912231+0000","last_change":"2024-05-09T08:27:55.822832+0000","last_active":"2024-05-09T08:27:57.912231+0000","last_peered":"2024-05-09T08:27:57.912231+0000","last_clean":"2024-05-09T08:27:57.912231+0000","last_became_active":"2024-05-09T08:27:55.822391+0000","last_became_peered":"2024-05-09T08:27:55.822391+0000","last_unstale":"2024-05-09T08:27:57.912231+0000","last_undegraded":"2024-05-09T08:27:57.912231+0000","last_fullsized":"2024-05-09T08:27:57.912231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:00.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.258927+0000","last_change":"2024-05-09T08:27:54.791027+0000","last_active":"2024-05-09T08:27:59.258927+0000","last_peered":"2024-05-09T08:27:59.258927+0000","last_clean":"2024-05-09T08:27:59.258927+0000","last_became_active":"2024-05-09T08:27:54.790726+0000","last_became_peered":"2024-05-09T08:27:54.790726+0000","last_unstale":"2024-05-09T08:27:59.258927+0000","last_undegraded":"2024-05-09T08:27:59.258927+0000","last_fullsized":"2024-05-09T08:27:59.258927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:37.795710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.224733+0000","last_change":"2024-05-09T08:27:55.799349+0000","last_active":"2024-05-09T08:27:57.224733+0000","last_peered":"2024-05-09T08:27:57.224733+0000","last_clean":"2024-05-09T08:27:57.224733+0000","last_became_active":"2024-05-09T08:27:55.798931+0000","last_became_peered":"2024-05-09T08:27:55.798931+0000","last_unstale":"2024-05-09T08:27:57.224733+0000","last_undegraded":"2024-05-09T08:27:57.224733+0000","last_fullsized":"2024-05-09T08:27:57.224733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:15.824884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.261110+0000","last_change":"2024-05-09T08:27:54.792088+0000","last_active":"2024-05-09T08:27:59.261110+0000","last_peered":"2024-05-09T08:27:59.261110+0000","last_clean":"2024-05-09T08:27:59.261110+0000","last_became_active":"2024-05-09T08:27:54.791526+0000","last_became_peered":"2024-05-09T08:27:54.791526+0000","last_unstale":"2024-05-09T08:27:59.261110+0000","last_undegraded":"2024-05-09T08:27:59.261110+0000","last_fullsized":"2024-05-09T08:27:59.261110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:11.494625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906651+0000","last_change":"2024-05-09T08:27:55.823258+0000","last_active":"2024-05-09T08:27:57.906651+0000","last_peered":"2024-05-09T08:27:57.906651+0000","last_clean":"2024-05-09T08:27:57.906651+0000","last_became_active":"2024-05-09T08:27:55.822923+0000","last_became_peered":"2024-05-09T08:27:55.822923+0000","last_unstale":"2024-05-09T08:27:57.906651+0000","last_undegraded":"2024-05-09T08:27:57.906651+0000","last_fullsized":"2024-05-09T08:27:57.906651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:17.481997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780714+0000","last_change":"2024-05-09T08:27:54.778865+0000","last_active":"2024-05-09T08:27:56.780714+0000","last_peered":"2024-05-09T08:27:56.780714+0000","last_clean":"2024-05-09T08:27:56.780714+0000","last_became_active":"2024-05-09T08:27:54.778494+0000","last_became_peered":"2024-05-09T08:27:54.778494+0000","last_unstale":"2024-05-09T08:27:56.780714+0000","last_undegraded":"2024-05-09T08:27:56.780714+0000","last_fullsized":"2024-05-09T08:27:56.780714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:19.632591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226515+0000","last_change":"2024-05-09T08:27:55.829385+0000","last_active":"2024-05-09T08:27:57.226515+0000","last_peered":"2024-05-09T08:27:57.226515+0000","last_clean":"2024-05-09T08:27:57.226515+0000","last_became_active":"2024-05-09T08:27:55.829045+0000","last_became_peered":"2024-05-09T08:27:55.829045+0000","last_unstale":"2024-05-09T08:27:57.226515+0000","last_undegraded":"2024-05-09T08:27:57.226515+0000","last_fullsized":"2024-05-09T08:27:57.226515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.171868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.257803+0000","last_change":"2024-05-09T08:27:54.770934+0000","last_active":"2024-05-09T08:27:59.257803+0000","last_peered":"2024-05-09T08:27:59.257803+0000","last_clean":"2024-05-09T08:27:59.257803+0000","last_became_active":"2024-05-09T08:27:54.770597+0000","last_became_peered":"2024-05-09T08:27:54.770597+0000","last_unstale":"2024-05-09T08:27:59.257803+0000","last_undegraded":"2024-05-09T08:27:59.257803+0000","last_fullsized":"2024-05-09T08:27:59.257803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:20.722491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387712+0000","last_change":"2024-05-09T08:27:55.804210+0000","last_active":"2024-05-09T08:27:57.387712+0000","last_peered":"2024-05-09T08:27:57.387712+0000","last_clean":"2024-05-09T08:27:57.387712+0000","last_became_active":"2024-05-09T08:27:55.803870+0000","last_became_peered":"2024-05-09T08:27:55.803870+0000","last_unstale":"2024-05-09T08:27:57.387712+0000","last_undegraded":"2024-05-09T08:27:57.387712+0000","last_fullsized":"2024-05-09T08:27:57.387712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:46.593084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:28:00.270724+0000","last_change":"2024-05-09T08:27:54.775110+0000","last_active":"2024-05-09T08:28:00.270724+0000","last_peered":"2024-05-09T08:28:00.270724+0000","last_clean":"2024-05-09T08:28:00.270724+0000","last_became_active":"2024-05-09T08:27:54.774812+0000","last_became_peered":"2024-05-09T08:27:54.774812+0000","last_unstale":"2024-05-09T08:28:00.270724+0000","last_undegraded":"2024-05-09T08:28:00.270724+0000","last_fullsized":"2024-05-09T08:28:00.270724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:51.955284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387809+0000","last_change":"2024-05-09T08:27:55.804766+0000","last_active":"2024-05-09T08:27:57.387809+0000","last_peered":"2024-05-09T08:27:57.387809+0000","last_clean":"2024-05-09T08:27:57.387809+0000","last_became_active":"2024-05-09T08:27:55.803711+0000","last_became_peered":"2024-05-09T08:27:55.803711+0000","last_unstale":"2024-05-09T08:27:57.387809+0000","last_undegraded":"2024-05-09T08:27:57.387809+0000","last_fullsized":"2024-05-09T08:27:57.387809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:46.683112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.907030+0000","last_change":"2024-05-09T08:27:54.786594+0000","last_active":"2024-05-09T08:27:57.907030+0000","last_peered":"2024-05-09T08:27:57.907030+0000","last_clean":"2024-05-09T08:27:57.907030+0000","last_became_active":"2024-05-09T08:27:54.786329+0000","last_became_peered":"2024-05-09T08:27:54.786329+0000","last_unstale":"2024-05-09T08:27:57.907030+0000","last_undegraded":"2024-05-09T08:27:57.907030+0000","last_fullsized":"2024-05-09T08:27:57.907030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:09:43.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:59.389719+0000","last_change":"2024-05-09T08:27:55.825679+0000","last_active":"2024-05-09T08:27:59.389719+0000","last_peered":"2024-05-09T08:27:59.389719+0000","last_clean":"2024-05-09T08:27:59.389719+0000","last_became_active":"2024-05-09T08:27:55.825294+0000","last_became_peered":"2024-05-09T08:27:55.825294+0000","last_unstale":"2024-05-09T08:27:59.389719+0000","last_undegraded":"2024-05-09T08:27:59.389719+0000","last_fullsized":"2024-05-09T08:27:59.389719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:34:07.114046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910694+0000","last_change":"2024-05-09T08:27:54.766960+0000","last_active":"2024-05-09T08:27:57.910694+0000","last_peered":"2024-05-09T08:27:57.910694+0000","last_clean":"2024-05-09T08:27:57.910694+0000","last_became_active":"2024-05-09T08:27:54.765693+0000","last_became_peered":"2024-05-09T08:27:54.765693+0000","last_unstale":"2024-05-09T08:27:57.910694+0000","last_undegraded":"2024-05-09T08:27:57.910694+0000","last_fullsized":"2024-05-09T08:27:57.910694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:29:50.080788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.387721+0000","last_change":"2024-05-09T08:27:55.806802+0000","last_active":"2024-05-09T08:27:57.387721+0000","last_peered":"2024-05-09T08:27:57.387721+0000","last_clean":"2024-05-09T08:27:57.387721+0000","last_became_active":"2024-05-09T08:27:55.806421+0000","last_became_peered":"2024-05-09T08:27:55.806421+0000","last_unstale":"2024-05-09T08:27:57.387721+0000","last_undegraded":"2024-05-09T08:27:57.387721+0000","last_fullsized":"2024-05-09T08:27:57.387721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:45.673043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906660+0000","last_change":"2024-05-09T08:27:55.799360+0000","last_active":"2024-05-09T08:27:57.906660+0000","last_peered":"2024-05-09T08:27:57.906660+0000","last_clean":"2024-05-09T08:27:57.906660+0000","last_became_active":"2024-05-09T08:27:55.799018+0000","last_became_peered":"2024-05-09T08:27:55.799018+0000","last_unstale":"2024-05-09T08:27:57.906660+0000","last_undegraded":"2024-05-09T08:27:57.906660+0000","last_fullsized":"2024-05-09T08:27:57.906660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:54.767546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780412+0000","last_change":"2024-05-09T08:27:55.803385+0000","last_active":"2024-05-09T08:27:56.780412+0000","last_peered":"2024-05-09T08:27:56.780412+0000","last_clean":"2024-05-09T08:27:56.780412+0000","last_became_active":"2024-05-09T08:27:55.803031+0000","last_became_peered":"2024-05-09T08:27:55.803031+0000","last_unstale":"2024-05-09T08:27:56.780412+0000","last_undegraded":"2024-05-09T08:27:56.780412+0000","last_fullsized":"2024-05-09T08:27:56.780412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:47.750225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780405+0000","last_change":"2024-05-09T08:27:55.825634+0000","last_active":"2024-05-09T08:27:56.780405+0000","last_peered":"2024-05-09T08:27:56.780405+0000","last_clean":"2024-05-09T08:27:56.780405+0000","last_became_active":"2024-05-09T08:27:55.825330+0000","last_became_peered":"2024-05-09T08:27:55.825330+0000","last_unstale":"2024-05-09T08:27:56.780405+0000","last_undegraded":"2024-05-09T08:27:56.780405+0000","last_fullsized":"2024-05-09T08:27:56.780405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:21:18.166324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.911087+0000","last_change":"2024-05-09T08:27:55.818372+0000","last_active":"2024-05-09T08:27:57.911087+0000","last_peered":"2024-05-09T08:27:57.911087+0000","last_clean":"2024-05-09T08:27:57.911087+0000","last_became_active":"2024-05-09T08:27:55.817982+0000","last_became_peered":"2024-05-09T08:27:55.817982+0000","last_unstale":"2024-05-09T08:27:57.911087+0000","last_undegraded":"2024-05-09T08:27:57.911087+0000","last_fullsized":"2024-05-09T08:27:57.911087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:16.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.909769+0000","last_change":"2024-05-09T08:27:55.820509+0000","last_active":"2024-05-09T08:27:57.909769+0000","last_peered":"2024-05-09T08:27:57.909769+0000","last_clean":"2024-05-09T08:27:57.909769+0000","last_became_active":"2024-05-09T08:27:55.820135+0000","last_became_peered":"2024-05-09T08:27:55.820135+0000","last_unstale":"2024-05-09T08:27:57.909769+0000","last_undegraded":"2024-05-09T08:27:57.909769+0000","last_fullsized":"2024-05-09T08:27:57.909769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:28.691640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906864+0000","last_change":"2024-05-09T08:27:55.801038+0000","last_active":"2024-05-09T08:27:57.906864+0000","last_peered":"2024-05-09T08:27:57.906864+0000","last_clean":"2024-05-09T08:27:57.906864+0000","last_became_active":"2024-05-09T08:27:55.800700+0000","last_became_peered":"2024-05-09T08:27:55.800700+0000","last_unstale":"2024-05-09T08:27:57.906864+0000","last_undegraded":"2024-05-09T08:27:57.906864+0000","last_fullsized":"2024-05-09T08:27:57.906864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:44:42.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:56.780866+0000","last_change":"2024-05-09T08:27:55.802266+0000","last_active":"2024-05-09T08:27:56.780866+0000","last_peered":"2024-05-09T08:27:56.780866+0000","last_clean":"2024-05-09T08:27:56.780866+0000","last_became_active":"2024-05-09T08:27:55.801670+0000","last_became_peered":"2024-05-09T08:27:55.801670+0000","last_unstale":"2024-05-09T08:27:56.780866+0000","last_undegraded":"2024-05-09T08:27:56.780866+0000","last_fullsized":"2024-05-09T08:27:56.780866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:53.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.227177+0000","last_change":"2024-05-09T08:27:55.820107+0000","last_active":"2024-05-09T08:27:57.227177+0000","last_peered":"2024-05-09T08:27:57.227177+0000","last_clean":"2024-05-09T08:27:57.227177+0000","last_became_active":"2024-05-09T08:27:55.819748+0000","last_became_peered":"2024-05-09T08:27:55.819748+0000","last_unstale":"2024-05-09T08:27:57.227177+0000","last_undegraded":"2024-05-09T08:27:57.227177+0000","last_fullsized":"2024-05-09T08:27:57.227177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:08:04.311374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906430+0000","last_change":"2024-05-09T08:27:55.821625+0000","last_active":"2024-05-09T08:27:57.906430+0000","last_peered":"2024-05-09T08:27:57.906430+0000","last_clean":"2024-05-09T08:27:57.906430+0000","last_became_active":"2024-05-09T08:27:55.821291+0000","last_became_peered":"2024-05-09T08:27:55.821291+0000","last_unstale":"2024-05-09T08:27:57.906430+0000","last_undegraded":"2024-05-09T08:27:57.906430+0000","last_fullsized":"2024-05-09T08:27:57.906430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:40.017767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.226118+0000","last_change":"2024-05-09T08:27:55.818212+0000","last_active":"2024-05-09T08:27:57.226118+0000","last_peered":"2024-05-09T08:27:57.226118+0000","last_clean":"2024-05-09T08:27:57.226118+0000","last_became_active":"2024-05-09T08:27:55.817874+0000","last_became_peered":"2024-05-09T08:27:55.817874+0000","last_unstale":"2024-05-09T08:27:57.226118+0000","last_undegraded":"2024-05-09T08:27:57.226118+0000","last_fullsized":"2024-05-09T08:27:57.226118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:20.189942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906627+0000","last_change":"2024-05-09T08:27:55.820128+0000","last_active":"2024-05-09T08:27:57.906627+0000","last_peered":"2024-05-09T08:27:57.906627+0000","last_clean":"2024-05-09T08:27:57.906627+0000","last_became_active":"2024-05-09T08:27:55.819717+0000","last_became_peered":"2024-05-09T08:27:55.819717+0000","last_unstale":"2024-05-09T08:27:57.906627+0000","last_undegraded":"2024-05-09T08:27:57.906627+0000","last_fullsized":"2024-05-09T08:27:57.906627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:48.324777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.910235+0000","last_change":"2024-05-09T08:27:55.820639+0000","last_active":"2024-05-09T08:27:57.910235+0000","last_peered":"2024-05-09T08:27:57.910235+0000","last_clean":"2024-05-09T08:27:57.910235+0000","last_became_active":"2024-05-09T08:27:55.820116+0000","last_became_peered":"2024-05-09T08:27:55.820116+0000","last_unstale":"2024-05-09T08:27:57.910235+0000","last_undegraded":"2024-05-09T08:27:57.910235+0000","last_fullsized":"2024-05-09T08:27:57.910235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:35.099440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-09T08:27:57.906516+0000","last_change":"2024-05-09T08:27:55.804807+0000","last_active":"2024-05-09T08:27:57.906516+0000","last_peered":"2024-05-09T08:27:57.906516+0000","last_clean":"2024-05-09T08:27:57.906516+0000","last_became_active":"2024-05-09T08:27:55.804410+0000","last_became_peered":"2024-05-09T08:27:55.804410+0000","last_unstale":"2024-05-09T08:27:57.906516+0000","last_undegraded":"2024-05-09T08:27:57.906516+0000","last_fullsized":"2024-05-09T08:27:57.906516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:03.796554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280679,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5613,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":131,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":23048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":442733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1035709,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1034819,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":443629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":445113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":1033995,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:28:05.597 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T08:28:05.597 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T08:28:05.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T08:28:05.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:05 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3678737078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:05.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:05 smithi160 ceph-mon[34070]: pgmap v320: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-05-09T08:28:05.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:05 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3678737078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:05.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:05 smithi188 ceph-mon[34645]: pgmap v320: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-05-09T08:28:06.089 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:06.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:06 smithi133 ceph-mon[29610]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:06.560 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:06 smithi133 ceph-mon[29610]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:06.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:06 smithi160 ceph-mon[34070]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:06.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:06 smithi160 ceph-mon[34070]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:06.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:06 smithi188 ceph-mon[34645]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:06.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:06 smithi188 ceph-mon[34645]: from='client.24704 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:28:07.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:07 smithi160 ceph-mon[34070]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-05-09T08:28:07.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:07 smithi133 ceph-mon[29610]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-05-09T08:28:07.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:07 smithi188 ceph-mon[34645]: pgmap v321: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-05-09T08:28:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 -- 172.21.15.133:0/2527455948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1035a0 msgr2=0x7f0a4c103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 --2- 172.21.15.133:0/2527455948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c103a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a380099c0 tx=0x7f0a38009cd0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 -- 172.21.15.133:0/2527455948 shutdown_connections 2024-05-09T08:28:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 --2- 172.21.15.133:0/2527455948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c103f40 0x7f0a4c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 --2- 172.21.15.133:0/2527455948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c1023a0 0x7f0a4c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 --2- 172.21.15.133:0/2527455948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c103a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 -- 172.21.15.133:0/2527455948 >> 172.21.15.133:0/2527455948 conn(0x7f0a4c0fd940 msgr2=0x7f0a4c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 -- 172.21.15.133:0/2527455948 shutdown_connections 2024-05-09T08:28:08.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.284+0000 7f0a53a00700 1 -- 172.21.15.133:0/2527455948 wait complete. 2024-05-09T08:28:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.285+0000 7f0a53a00700 1 Processor -- start 2024-05-09T08:28:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.285+0000 7f0a53a00700 1 -- start start 2024-05-09T08:28:08.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1023a0 0x7f0a4c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10dfd0 con 0x7f0a4c1023a0 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10de50 con 0x7f0a4c103f40 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a53a00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10e150 con 0x7f0a4c1035a0 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1023a0 0x7f0a4c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:35988/0 (socket says 172.21.15.133:35988) 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 -- 172.21.15.133:0/3030276604 learned_addr learned my addr 172.21.15.133:0/3030276604 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a4be00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 -- 172.21.15.133:0/3030276604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c1035a0 msgr2=0x7f0a4c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 -- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1023a0 msgr2=0x7f0a4c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1023a0 0x7f0a4c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.286+0000 7f0a51a00700 1 -- 172.21.15.133:0/3030276604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c1a2390 con 0x7f0a4c103f40 2024-05-09T08:28:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a51a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400ef30 tx=0x7f0a4400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440092e0 con 0x7f0a4c103f40 2024-05-09T08:28:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44004bb0 con 0x7f0a4c103f40 2024-05-09T08:28:08.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1a2620 con 0x7f0a4c103f40 2024-05-09T08:28:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44004280 con 0x7f0a4c103f40 2024-05-09T08:28:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.287+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1a2bb0 con 0x7f0a4c103f40 2024-05-09T08:28:08.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.288+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c050890 con 0x7f0a4c103f40 2024-05-09T08:28:08.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.290+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0a4401b030 con 0x7f0a4c103f40 2024-05-09T08:28:08.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.292+0000 7f0a49600700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 0x7f0a340a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.292+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0a440ceaf0 con 0x7f0a4c103f40 2024-05-09T08:28:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.292+0000 7f0a51000700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 0x7f0a340a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:08.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.293+0000 7f0a51000700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 0x7f0a340a52a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c103400 tx=0x7f0a40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.296+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a44022020 con 0x7f0a4c103f40 2024-05-09T08:28:08.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.608+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0a4c1a30b0 con 0x7f0a4c103f40 2024-05-09T08:28:08.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.616+0000 7f0a49600700 1 -- 172.21.15.133:0/3030276604 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f0a4c1a30b0 con 0x7f0a4c103f40 2024-05-09T08:28:08.616 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-05-09T08:28:08.616 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAYiTxmuj1sJBAAn1RpL0nrW6mSzyb+cuozlA== 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 msgr2=0x7f0a340a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 0x7f0a340a52a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c103400 tx=0x7f0a40009260 comp rx=0 tx=0).stop 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 msgr2=0x7f0a4c1a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400ef30 tx=0x7f0a4400c3a0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 shutdown_connections 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a4c103f40 0x7f0a4c1a1cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a4c1023a0 0x7f0a4c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f0a340a2de0 0x7f0a340a52a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 --2- 172.21.15.133:0/3030276604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a4c1035a0 0x7f0a4c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:08.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 >> 172.21.15.133:0/3030276604 conn(0x7f0a4c0fd940 msgr2=0x7f0a4c074550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:08.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 shutdown_connections 2024-05-09T08:28:08.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:08.619+0000 7f0a53a00700 1 -- 172.21.15.133:0/3030276604 wait complete. 2024-05-09T08:28:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:08 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3030276604' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:08 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:08 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T08:28:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:08 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3030276604' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:08 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:08.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:08 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T08:28:09.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:08 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3030276604' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:09.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:08 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T08:28:09.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:08 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T08:28:09.524 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:28:09.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T08:28:09.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T08:28:09.655 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T08:28:09.655 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T08:28:09.655 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T08:28:09.767 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:09 smithi133 ceph-mon[29610]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-05-09T08:28:09.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:09 smithi188 ceph-mon[34645]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-05-09T08:28:10.136 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:10.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:09 smithi160 ceph-mon[34070]: pgmap v322: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-05-09T08:28:11.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:11 smithi133 ceph-mon[29610]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-05-09T08:28:11.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:11 smithi188 ceph-mon[34645]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-05-09T08:28:12.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:11 smithi160 ceph-mon[34070]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-05-09T08:28:12.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 -- 172.21.15.133:0/1060756696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1024074e70 msgr2=0x7f102410e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 --2- 172.21.15.133:0/1060756696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1024074e70 0x7f102410e7e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 -- 172.21.15.133:0/1060756696 shutdown_connections 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 --2- 172.21.15.133:0/1060756696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f102410ed20 0x7f1024111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 --2- 172.21.15.133:0/1060756696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074550 0x7f1024074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 --2- 172.21.15.133:0/1060756696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1024074e70 0x7f102410e7e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 -- 172.21.15.133:0/1060756696 >> 172.21.15.133:0/1060756696 conn(0x7f1024079480 msgr2=0x7f1024079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 -- 172.21.15.133:0/1060756696 shutdown_connections 2024-05-09T08:28:12.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.421+0000 7f1029800700 1 -- 172.21.15.133:0/1060756696 wait complete. 2024-05-09T08:28:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 Processor -- start 2024-05-09T08:28:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 -- start start 2024-05-09T08:28:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 0x7f10241a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:12.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102410ed20 0x7f102419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1024114880 con 0x7f1024074550 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1024114700 con 0x7f102410ed20 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.423+0000 7f1029800700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1024114a00 con 0x7f1024074e70 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59606/0 (socket says 172.21.15.133:59606) 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 -- 172.21.15.133:0/306608168 learned_addr learned my addr 172.21.15.133:0/306608168 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022000700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 0x7f10241a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:12.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1023400700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102410ed20 0x7f102419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 -- 172.21.15.133:0/306608168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 msgr2=0x7f10241a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 0x7f10241a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 -- 172.21.15.133:0/306608168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102410ed20 msgr2=0x7f102419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102410ed20 0x7f102419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022a00700 1 -- 172.21.15.133:0/306608168 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102419b460 con 0x7f1024074550 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.424+0000 7f1022000700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 0x7f10241a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:12.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f1022a00700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9a0 tx=0x7f101400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400ca10 con 0x7f1024074550 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101400cbb0 con 0x7f1024074550 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014018680 con 0x7f1024074550 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102419b750 con 0x7f1024074550 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.425+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10241a79e0 con 0x7f1024074550 2024-05-09T08:28:12.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.426+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1024075c40 con 0x7f1024074550 2024-05-09T08:28:12.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.430+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1014010430 con 0x7f1024074550 2024-05-09T08:28:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.431+0000 7f101b400700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 0x7f10080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.431+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f10140cf240 con 0x7f1024074550 2024-05-09T08:28:12.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.432+0000 7f1022000700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 0x7f10080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:12.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.432+0000 7f1022000700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 0x7f10080a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f102419c310 tx=0x7f1010005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:12.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.436+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f101409c170 con 0x7f1024074550 2024-05-09T08:28:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.724+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f102419bff0 con 0x7f1024074550 2024-05-09T08:28:12.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.734+0000 7f101b400700 1 -- 172.21.15.133:0/306608168 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263207 (secure 0 0 0) 0x7f101409ba70 con 0x7f1024074550 2024-05-09T08:28:12.738 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 msgr2=0x7f10080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 0x7f10080a51f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f102419c310 tx=0x7f1010005fd0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 msgr2=0x7f10241a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9a0 tx=0x7f101400ecb0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 shutdown_connections 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f102410ed20 0x7f102419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1024074550 0x7f10241a0a60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f10080a2d30 0x7f10080a51f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 --2- 172.21.15.133:0/306608168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1024074e70 0x7f10241a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 >> 172.21.15.133:0/306608168 conn(0x7f1024079480 msgr2=0x7f10240ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 shutdown_connections 2024-05-09T08:28:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:12.739+0000 7f1029800700 1 -- 172.21.15.133:0/306608168 wait complete. 2024-05-09T08:28:13.658 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":2058981411},{"type":"v1","addr":"172.21.15.133:6801","nonce":2058981411}]},"active_addr":"172.21.15.133:6801/2058981411","active_change":"2024-05-09T08:19:56.515822+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.133:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3360446484}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":274371222}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":648550069}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":454568591}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2143407685}]}]} 2024-05-09T08:28:13.665 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T08:28:13.665 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T08:28:13.665 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T08:28:13.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:13 smithi133 ceph-mon[29610]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-05-09T08:28:13.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:13 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/306608168' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T08:28:13.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:13 smithi188 ceph-mon[34645]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-05-09T08:28:13.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:13 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/306608168' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T08:28:14.095 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:13 smithi160 ceph-mon[34070]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-05-09T08:28:14.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:13 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/306608168' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T08:28:15.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:15 smithi133 ceph-mon[29610]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T08:28:15.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:15 smithi188 ceph-mon[34645]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T08:28:16.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:15 smithi160 ceph-mon[34070]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.317+0000 7f78c8200700 1 -- 172.21.15.133:0/103112985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074550 msgr2=0x7f78c0074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 --2- 172.21.15.133:0/103112985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074550 0x7f78c0074930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009990 tx=0x7f78b4009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 -- 172.21.15.133:0/103112985 shutdown_connections 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 --2- 172.21.15.133:0/103112985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c010ed20 0x7f78c0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 --2- 172.21.15.133:0/103112985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074550 0x7f78c0074930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 --2- 172.21.15.133:0/103112985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c0074e70 0x7f78c010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 -- 172.21.15.133:0/103112985 >> 172.21.15.133:0/103112985 conn(0x7f78c0079480 msgr2=0x7f78c0079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:16.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 -- 172.21.15.133:0/103112985 shutdown_connections 2024-05-09T08:28:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.318+0000 7f78c8200700 1 -- 172.21.15.133:0/103112985 wait complete. 2024-05-09T08:28:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 Processor -- start 2024-05-09T08:28:16.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 -- start start 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 0x7f78c019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 0x7f78c019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78c0114870 con 0x7f78c0074e70 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01146f0 con 0x7f78c0074550 2024-05-09T08:28:16.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.320+0000 7f78c8200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78c01149f0 con 0x7f78c010ed20 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c4e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c4e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54116/0 (socket says 172.21.15.133:54116) 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 0x7f78c019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c4e00700 1 -- 172.21.15.133:0/3120226595 learned_addr learned my addr 172.21.15.133:0/3120226595 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c5800700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 0x7f78c019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 -- 172.21.15.133:0/3120226595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 msgr2=0x7f78c019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 0x7f78c019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 -- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 msgr2=0x7f78c019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 -- 172.21.15.133:0/3120226595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c019fd50 con 0x7f78c010ed20 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c5800700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 0x7f78c019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:16.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c4e00700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.321+0000 7f78c6200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 0x7f78c019f610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78b800efb0 tx=0x7f78b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.322+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003a00 con 0x7f78c010ed20 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.322+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c01a0040 con 0x7f78c010ed20 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.322+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b8003ba0 con 0x7f78c010ed20 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.322+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8004850 con 0x7f78c010ed20 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.322+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c01a78c0 con 0x7f78c010ed20 2024-05-09T08:28:16.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.323+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c0069010 con 0x7f78c010ed20 2024-05-09T08:28:16.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.325+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f78b8012050 con 0x7f78c010ed20 2024-05-09T08:28:16.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.326+0000 7f78be000700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 0x7f78a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:16.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.326+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f78b80ce910 con 0x7f78c010ed20 2024-05-09T08:28:16.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.327+0000 7f78c5800700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 0x7f78a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:16.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.327+0000 7f78c5800700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 0x7f78a80a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:16.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.331+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78b8022060 con 0x7f78c010ed20 2024-05-09T08:28:16.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.556+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78c002db80 con 0x7f78c010ed20 2024-05-09T08:28:16.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.558+0000 7f78be000700 1 -- 172.21.15.133:0/3120226595 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22287 (secure 0 0 0) 0x7f78b8010040 con 0x7f78c010ed20 2024-05-09T08:28:16.559 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:16.559 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":86,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:56.775532+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 msgr2=0x7f78a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 0x7f78a80a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b400b260 comp rx=0 tx=0).stop 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 msgr2=0x7f78c019f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 0x7f78c019f610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78b800efb0 tx=0x7f78b800c3a0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 shutdown_connections 2024-05-09T08:28:16.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78a80a2cb0 0x7f78a80a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78c010ed20 0x7f78c019f610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78c0074550 0x7f78c019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 --2- 172.21.15.133:0/3120226595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c0074e70 0x7f78c019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 >> 172.21.15.133:0/3120226595 conn(0x7f78c0079480 msgr2=0x7f78c00ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 shutdown_connections 2024-05-09T08:28:16.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:16.561+0000 7f78c8200700 1 -- 172.21.15.133:0/3120226595 wait complete. 2024-05-09T08:28:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:16 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3120226595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:17.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:16 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3120226595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:17.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:16 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3120226595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:17.461 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T08:28:17.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-09T08:28:17.913 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:18.059 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:17 smithi133 ceph-mon[29610]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T08:28:18.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:17 smithi160 ceph-mon[34070]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T08:28:18.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:17 smithi188 ceph-mon[34645]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-05-09T08:28:19.909 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:19 smithi133 ceph-mon[29610]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-09T08:28:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 -- 172.21.15.133:0/1835379201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770108950 msgr2=0x7f5770108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 --2- 172.21.15.133:0/1835379201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770108950 0x7f5770108dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5758009990 tx=0x7f5758009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 -- 172.21.15.133:0/1835379201 shutdown_connections 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 --2- 172.21.15.133:0/1835379201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5770109310 0x7f5770115820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 --2- 172.21.15.133:0/1835379201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f577010e950 0x7f577010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 --2- 172.21.15.133:0/1835379201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770108950 0x7f5770108dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 -- 172.21.15.133:0/1835379201 >> 172.21.15.133:0/1835379201 conn(0x7f5770075af0 msgr2=0x7f5770107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 -- 172.21.15.133:0/1835379201 shutdown_connections 2024-05-09T08:28:20.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.075+0000 7f5776e00700 1 -- 172.21.15.133:0/1835379201 wait complete. 2024-05-09T08:28:20.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.076+0000 7f5776e00700 1 Processor -- start 2024-05-09T08:28:20.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.076+0000 7f5776e00700 1 -- start start 2024-05-09T08:28:20.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:20.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770109310 0x7f577019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f577010e950 0x7f57701a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5770118ce0 con 0x7f5770109310 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5770118b60 con 0x7f5770108950 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5776e00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5770118e60 con 0x7f577010e950 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5767e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770109310 0x7f577019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:45494/0 (socket says 172.21.15.133:45494) 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 -- 172.21.15.133:0/2646624123 learned_addr learned my addr 172.21.15.133:0/2646624123 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f5774e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f577010e950 0x7f57701a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 -- 172.21.15.133:0/2646624123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f577010e950 msgr2=0x7f57701a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f577010e950 0x7f57701a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 -- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770109310 msgr2=0x7f577019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:20.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770109310 0x7f577019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.077+0000 7f576fe00700 1 -- 172.21.15.133:0/2646624123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57701a40c0 con 0x7f5770108950 2024-05-09T08:28:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f576fe00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f576000e9a0 tx=0x7f576000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576000ca10 con 0x7f5770108950 2024-05-09T08:28:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576000cbb0 con 0x7f5770108950 2024-05-09T08:28:20.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57701a43b0 con 0x7f5770108950 2024-05-09T08:28:20.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760018680 con 0x7f5770108950 2024-05-09T08:28:20.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.078+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57701abc30 con 0x7f5770108950 2024-05-09T08:28:20.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.079+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5770069010 con 0x7f5770108950 2024-05-09T08:28:20.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.081+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5760010430 con 0x7f5770108950 2024-05-09T08:28:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.083+0000 7f576d600700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 0x7f57540a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.083+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f57600cf730 con 0x7f5770108950 2024-05-09T08:28:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.083+0000 7f5767e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 0x7f57540a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:20.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.084+0000 7f5767e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 0x7f57540a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f57580098e0 tx=0x7f5758005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:20.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.085+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f576009c5e0 con 0x7f5770108950 2024-05-09T08:28:20.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:19 smithi160 ceph-mon[34070]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-09T08:28:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:19 smithi188 ceph-mon[34645]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-05-09T08:28:20.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.310+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f577002db80 con 0x7f5770108950 2024-05-09T08:28:20.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.313+0000 7f576d600700 1 -- 172.21.15.133:0/2646624123 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22287 (secure 0 0 0) 0x7f576009bee0 con 0x7f5770108950 2024-05-09T08:28:20.314 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:28:20.314 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":86,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:27:56.775532+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:28:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 msgr2=0x7f57540a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 0x7f57540a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f57580098e0 tx=0x7f5758005960 comp rx=0 tx=0).stop 2024-05-09T08:28:20.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 msgr2=0x7f577019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f576000e9a0 tx=0x7f576000ecb0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 shutdown_connections 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f57540a2d30 0x7f57540a51f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f577010e950 0x7f57701a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5770108950 0x7f577019f0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 --2- 172.21.15.133:0/2646624123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5770109310 0x7f577019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 >> 172.21.15.133:0/2646624123 conn(0x7f5770075af0 msgr2=0x7f5770076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 shutdown_connections 2024-05-09T08:28:20.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:20.316+0000 7f5776e00700 1 -- 172.21.15.133:0/2646624123 wait complete. 2024-05-09T08:28:20.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:20 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2646624123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:21.048 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:20 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2646624123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:21.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T08:28:21.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T08:28:21.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T08:28:21.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T08:28:21.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T08:28:21.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T08:28:21.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T08:28:21.130 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T08:28:21.130 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T08:28:21.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T08:28:21.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T08:28:21.132 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T08:28:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:20 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2646624123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:28:22.110 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:22.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:21 smithi160 ceph-mon[34070]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T08:28:22.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:21 smithi133 ceph-mon[29610]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T08:28:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:21 smithi188 ceph-mon[34645]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-05-09T08:28:22.261 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:23.984 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:23 smithi133 ceph-mon[29610]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T08:28:24.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:23 smithi160 ceph-mon[34070]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T08:28:24.173 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:24.174 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:24.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:23 smithi188 ceph-mon[34645]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 -- 172.21.15.133:0/2111270909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c1083a0 msgr2=0x7f9e5c108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 --2- 172.21.15.133:0/2111270909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c1083a0 0x7f9e5c108780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009990 tx=0x7f9e4c009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 -- 172.21.15.133:0/2111270909 shutdown_connections 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 --2- 172.21.15.133:0/2111270909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e5c06a9d0 0x7f9e5c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 --2- 172.21.15.133:0/2111270909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c1083a0 0x7f9e5c108780 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 --2- 172.21.15.133:0/2111270909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5c06a030 0x7f9e5c06a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 -- 172.21.15.133:0/2111270909 >> 172.21.15.133:0/2111270909 conn(0x7f9e5c0781d0 msgr2=0x7f9e5c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 -- 172.21.15.133:0/2111270909 shutdown_connections 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.316+0000 7f9e62600700 1 -- 172.21.15.133:0/2111270909 wait complete. 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.317+0000 7f9e62600700 1 Processor -- start 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.317+0000 7f9e62600700 1 -- start start 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.317+0000 7f9e62600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 0x7f9e5c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.317+0000 7f9e62600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e5c06a9d0 0x7f9e5c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e62600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5c1083a0 0x7f9e5c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e62600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c112650 con 0x7f9e5c06a030 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e62600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1124d0 con 0x7f9e5c06a9d0 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e62600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c1127d0 con 0x7f9e5c1083a0 2024-05-09T08:28:25.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 0x7f9e5c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 0x7f9e5c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56908/0 (socket says 172.21.15.133:56908) 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 -- 172.21.15.133:0/3049439450 learned_addr learned my addr 172.21.15.133:0/3049439450 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5aa00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e5c06a9d0 0x7f9e5c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5be00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5c1083a0 0x7f9e5c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 -- 172.21.15.133:0/3049439450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5c1083a0 msgr2=0x7f9e5c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5c1083a0 0x7f9e5c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 -- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e5c06a9d0 msgr2=0x7f9e5c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e5c06a9d0 0x7f9e5c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c19fd40 con 0x7f9e5c06a030 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.318+0000 7f9e5b400700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 0x7f9e5c19ad30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005910 tx=0x7f9e4c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.319+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c027070 con 0x7f9e5c06a030 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.319+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c19ffd0 con 0x7f9e5c06a030 2024-05-09T08:28:25.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.319+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c004650 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.319+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00f460 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.319+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c1a78b0 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.320+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e5c10df90 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.322+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9e4c02a070 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.323+0000 7f9e53e00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9e400a2cb0 0x7f9e400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.323+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9e4c0d06f0 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.323+0000 7f9e5aa00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9e400a2cb0 0x7f9e400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e53e00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] conn(0x7f9e400ac490 0x7f9e400ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e400aef60 con 0x7f9e400ac490 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c0d0aa0 con 0x7f9e5c06a030 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e5be00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] conn(0x7f9e400ac490 0x7f9e400ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e5aa00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9e400a2cb0 0x7f9e400a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c103320 tx=0x7f9e48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:25.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.324+0000 7f9e5be00700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] conn(0x7f9e400ac490 0x7f9e400ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.325+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== osd.4 v2:172.21.15.160:6808/1804660056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e400aef60 con 0x7f9e400ac490 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.343+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 --> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e5c103a50 con 0x7f9e400ac490 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.344+0000 7f9e53e00700 1 -- 172.21.15.133:0/3049439450 <== osd.4 v2:172.21.15.160:6808/1804660056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9e5c103a50 con 0x7f9e400ac490 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] conn(0x7f9e400ac490 msgr2=0x7f9e400ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] conn(0x7f9e400ac490 0x7f9e400ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9e400a2cb0 msgr2=0x7f9e400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f9e400a2cb0 0x7f9e400a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c103320 tx=0x7f9e48009260 comp rx=0 tx=0).stop 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 msgr2=0x7f9e5c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 --2- 172.21.15.133:0/3049439450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e5c06a030 0x7f9e5c19ad30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005910 tx=0x7f9e4c00bab0 comp rx=0 tx=0).stop 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e5be00700 1 -- 172.21.15.133:0/3049439450 reap_dead start 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 shutdown_connections 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 >> 172.21.15.133:0/3049439450 conn(0x7f9e5c0781d0 msgr2=0x7f9e5c10d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 shutdown_connections 2024-05-09T08:28:25.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:25.345+0000 7f9e62600700 1 -- 172.21.15.133:0/3049439450 wait complete. 2024-05-09T08:28:25.933 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.934 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.934 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.934 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.938 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:25.945 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:26.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:25 smithi133 ceph-mon[29610]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:26.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:25 smithi160 ceph-mon[34070]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:26.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:25 smithi188 ceph-mon[34645]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 -- 172.21.15.133:0/1049237166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102d30 msgr2=0x7f850c10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 --2- 172.21.15.133:0/1049237166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102d30 0x7f850c10f220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f850400b0a0 tx=0x7f850400b3b0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 -- 172.21.15.133:0/1049237166 shutdown_connections 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 --2- 172.21.15.133:0/1049237166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102d30 0x7f850c10f220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 --2- 172.21.15.133:0/1049237166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f850c1083e0 0x7f850c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 --2- 172.21.15.133:0/1049237166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102370 0x7f850c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 -- 172.21.15.133:0/1049237166 >> 172.21.15.133:0/1049237166 conn(0x7f850c0794d0 msgr2=0x7f850c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 -- 172.21.15.133:0/1049237166 shutdown_connections 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.534+0000 7f8514000700 1 -- 172.21.15.133:0/1049237166 wait complete. 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.535+0000 7f8514000700 1 Processor -- start 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.535+0000 7f8514000700 1 -- start start 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102370 0x7f850c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102d30 0x7f850c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f850c1083e0 0x7f850c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1126a0 con 0x7f850c102370 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f850c112520 con 0x7f850c102d30 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8514000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f850c112820 con 0x7f850c1083e0 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8511600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102370 0x7f850c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8510c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102d30 0x7f850c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8511600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102370 0x7f850c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56918/0 (socket says 172.21.15.133:56918) 2024-05-09T08:28:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8511600700 1 -- 172.21.15.133:0/2783050886 learned_addr learned my addr 172.21.15.133:0/2783050886 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8512000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f850c1083e0 0x7f850c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8510c00700 1 -- 172.21.15.133:0/2783050886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f850c1083e0 msgr2=0x7f850c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.536+0000 7f8510c00700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f850c1083e0 0x7f850c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8510c00700 1 -- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102370 msgr2=0x7f850c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8510c00700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c102370 0x7f850c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8510c00700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c19fd90 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8510c00700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102d30 0x7f850c19b2c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e9f0 tx=0x7f84f800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f800c9c0 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f800cb60 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c1a0080 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8018680 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.537+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a7900 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.538+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f850c10f190 con 0x7f850c102d30 2024-05-09T08:28:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.540+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f84f8010430 con 0x7f850c102d30 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f850a000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84f40a2d30 0x7f84f40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f84f80cf660 con 0x7f850c102d30 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f8511600700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84f40a2d30 0x7f84f40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f850a000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] conn(0x7f84f40ac510 0x7f84f40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84f801bec0 con 0x7f84f40ac510 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f84f80cfa10 con 0x7f850c102d30 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.542+0000 7f8512000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] conn(0x7f84f40ac510 0x7f84f40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.543+0000 7f8511600700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84f40a2d30 0x7f84f40a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8500009ca0 tx=0x7f8500009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.543+0000 7f8512000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] conn(0x7f84f40ac510 0x7f84f40ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.544+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== osd.0 v2:172.21.15.133:6802/2872423449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84f801bec0 con 0x7f84f40ac510 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.562+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 --> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f850c103a90 con 0x7f84f40ac510 2024-05-09T08:28:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.562+0000 7f850a000700 1 -- 172.21.15.133:0/2783050886 <== osd.0 v2:172.21.15.133:6802/2872423449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f850c103a90 con 0x7f84f40ac510 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] conn(0x7f84f40ac510 msgr2=0x7f84f40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] conn(0x7f84f40ac510 0x7f84f40ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84f40a2d30 msgr2=0x7f84f40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84f40a2d30 0x7f84f40a51f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8500009ca0 tx=0x7f8500009260 comp rx=0 tx=0).stop 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102d30 msgr2=0x7f850c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 --2- 172.21.15.133:0/2783050886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f850c102d30 0x7f850c19b2c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e9f0 tx=0x7f84f800ed00 comp rx=0 tx=0).stop 2024-05-09T08:28:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8512000700 1 -- 172.21.15.133:0/2783050886 reap_dead start 2024-05-09T08:28:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 shutdown_connections 2024-05-09T08:28:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.563+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 >> 172.21.15.133:0/2783050886 conn(0x7f850c0794d0 msgr2=0x7f850c10d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.564+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 shutdown_connections 2024-05-09T08:28:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:26.564+0000 7f8514000700 1 -- 172.21.15.133:0/2783050886 wait complete. 2024-05-09T08:28:28.146 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:27 smithi133 ceph-mon[29610]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:28.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:27 smithi160 ceph-mon[34070]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:28.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:27 smithi188 ceph-mon[34645]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:29.444 INFO:teuthology.orchestra.run.smithi133.stdout:146028888123 2024-05-09T08:28:29.444 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T08:28:30.050 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:29 smithi133 ceph-mon[29610]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T08:28:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:29 smithi160 ceph-mon[34070]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T08:28:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:29 smithi188 ceph-mon[34645]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T08:28:32.104 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:31 smithi133 ceph-mon[29610]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:32.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:31 smithi160 ceph-mon[34070]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:32.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:31 smithi188 ceph-mon[34645]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:32.708 INFO:teuthology.orchestra.run.smithi133.stdout:34359738449 2024-05-09T08:28:32.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T08:28:33.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:33 smithi133 ceph-mon[29610]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:34.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:33 smithi160 ceph-mon[34070]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:34.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:33 smithi188 ceph-mon[34645]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:28:35.097 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:34 smithi133 ceph-mon[29610]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:35.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:34 smithi160 ceph-mon[34070]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:34 smithi188 ceph-mon[34645]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:37.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:37 smithi133 ceph-mon[29610]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:37 smithi160 ceph-mon[34070]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:38.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:37 smithi188 ceph-mon[34645]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 -- 172.21.15.133:0/820244667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c1025d0 msgr2=0x7f789c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 --2- 172.21.15.133:0/820244667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c1025d0 0x7f789c1029b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f788c009990 tx=0x7f788c009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 -- 172.21.15.133:0/820244667 shutdown_connections 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 --2- 172.21.15.133:0/820244667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f789c107660 0x7f789c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 --2- 172.21.15.133:0/820244667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c1025d0 0x7f789c1029b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 --2- 172.21.15.133:0/820244667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c102f80 0x7f789c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 -- 172.21.15.133:0/820244667 >> 172.21.15.133:0/820244667 conn(0x7f789c079480 msgr2=0x7f789c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 -- 172.21.15.133:0/820244667 shutdown_connections 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.399+0000 7f78a1c00700 1 -- 172.21.15.133:0/820244667 wait complete. 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.400+0000 7f78a1c00700 1 Processor -- start 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.400+0000 7f78a1c00700 1 -- start start 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 0x7f789c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c102f80 0x7f789c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f789c107660 0x7f789c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f789c112650 con 0x7f789c1025d0 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1124d0 con 0x7f789c102f80 2024-05-09T08:28:38.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f78a1c00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1127d0 con 0x7f789c107660 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f789aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 0x7f789c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f789aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 0x7f789c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40514/0 (socket says 172.21.15.133:40514) 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f789aa00700 1 -- 172.21.15.133:0/711331456 learned_addr learned my addr 172.21.15.133:0/711331456 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f789a000700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c102f80 0x7f789c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.401+0000 7f789b400700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f789c107660 0x7f789c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 -- 172.21.15.133:0/711331456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f789c107660 msgr2=0x7f789c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f789c107660 0x7f789c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 -- 172.21.15.133:0/711331456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c102f80 msgr2=0x7f789c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f789c102f80 0x7f789c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c199070 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f789aa00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 0x7f789c19e750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f788c00bae0 tx=0x7f788c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c026070 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c003d70 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c199300 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.402+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c021410 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.403+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c1a5620 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.403+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f789c062a70 con 0x7f789c1025d0 2024-05-09T08:28:38.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.407+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f788c029070 con 0x7f789c1025d0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.408+0000 7f7893400700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78800a2cb0 0x7f78800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f788c0d05c0 con 0x7f789c1025d0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f789a000700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78800a2cb0 0x7f78800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f7893400700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] conn(0x7f78800ac490 0x7f78800ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f788c023ec0 con 0x7f78800ac490 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f788c0d0970 con 0x7f789c1025d0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.409+0000 7f789b400700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] conn(0x7f78800ac490 0x7f78800ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.410+0000 7f789a000700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78800a2cb0 0x7f78800a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f789c101f60 tx=0x7f7888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.410+0000 7f789b400700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] conn(0x7f78800ac490 0x7f78800ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.411+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== osd.9 v2:172.21.15.133:6826/3729207054 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f788c023ec0 con 0x7f78800ac490 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.429+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 --> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f789c062ef0 con 0x7f78800ac490 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.430+0000 7f7893400700 1 -- 172.21.15.133:0/711331456 <== osd.9 v2:172.21.15.133:6826/3729207054 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f789c062ef0 con 0x7f78800ac490 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] conn(0x7f78800ac490 msgr2=0x7f78800ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] conn(0x7f78800ac490 0x7f78800ae8b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78800a2cb0 msgr2=0x7f78800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f78800a2cb0 0x7f78800a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f789c101f60 tx=0x7f7888009260 comp rx=0 tx=0).stop 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 msgr2=0x7f789c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 --2- 172.21.15.133:0/711331456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f789c1025d0 0x7f789c19e750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f788c00bae0 tx=0x7f788c00bb10 comp rx=0 tx=0).stop 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f789b400700 1 -- 172.21.15.133:0/711331456 reap_dead start 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 shutdown_connections 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 >> 172.21.15.133:0/711331456 conn(0x7f789c079480 msgr2=0x7f789c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 shutdown_connections 2024-05-09T08:28:38.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.431+0000 7f78a1c00700 1 -- 172.21.15.133:0/711331456 wait complete. 2024-05-09T08:28:38.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.505+0000 7fc505c00700 1 -- 172.21.15.133:0/800834362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5001025d0 msgr2=0x7fc5001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.505+0000 7fc505c00700 1 --2- 172.21.15.133:0/800834362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5001025d0 0x7fc5001029b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009990 tx=0x7fc4f0009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 -- 172.21.15.133:0/800834362 shutdown_connections 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 --2- 172.21.15.133:0/800834362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 0x7fc50010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 --2- 172.21.15.133:0/800834362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5001025d0 0x7fc5001029b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 --2- 172.21.15.133:0/800834362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc500102f80 0x7fc500106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 -- 172.21.15.133:0/800834362 >> 172.21.15.133:0/800834362 conn(0x7fc5000794d0 msgr2=0x7fc5000798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 -- 172.21.15.133:0/800834362 shutdown_connections 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.506+0000 7fc505c00700 1 -- 172.21.15.133:0/800834362 wait complete. 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 Processor -- start 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 -- start start 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5001025d0 0x7fc50019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 0x7fc50019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 0x7fc50019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001126a0 con 0x7fc500102f80 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc500112520 con 0x7fc5001025d0 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc505c00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc500112820 con 0x7fc500107690 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.507+0000 7fc4fe000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 0x7fc50019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 0x7fc50019b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40526/0 (socket says 172.21.15.133:40526) 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 -- 172.21.15.133:0/1267513447 learned_addr learned my addr 172.21.15.133:0/1267513447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fea00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5001025d0 0x7fc50019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4ff400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 0x7fc50019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 -- 172.21.15.133:0/1267513447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 msgr2=0x7fc50019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 0x7fc50019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 -- 172.21.15.133:0/1267513447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5001025d0 msgr2=0x7fc50019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5001025d0 0x7fc50019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50019fd40 con 0x7fc500102f80 2024-05-09T08:28:38.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fea00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5001025d0 0x7fc50019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4ff400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc500107690 0x7fc50019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.508+0000 7fc4fe000700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 0x7fc50019b270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e970 tx=0x7fc4ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.509+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00cbd0 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.509+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ec00cd70 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.509+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0056b0 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.509+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5001a0030 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.509+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5001a78b0 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.510+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5000758b0 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.513+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc4ec004750 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4f7400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc4e40a2cb0 0x7fc4e40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc4ec0cfb90 con 0x7fc500102f80 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4fea00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc4e40a2cb0 0x7fc4e40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4f7400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] conn(0x7fc4e40ac490 0x7fc4e40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:38.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4ec01ee50 con 0x7fc4e40ac490 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.514+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc4ec0cff40 con 0x7fc500102f80 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.515+0000 7fc4ff400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] conn(0x7fc4e40ac490 0x7fc4e40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.515+0000 7fc4fea00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc4e40a2cb0 0x7fc4e40a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006010 tx=0x7fc4f000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.515+0000 7fc4ff400700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] conn(0x7fc4e40ac490 0x7fc4e40ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.516+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== osd.3 v2:172.21.15.133:6810/2675627022 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4ec01ee50 con 0x7fc4e40ac490 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.534+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 --> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc500103a90 con 0x7fc4e40ac490 2024-05-09T08:28:38.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.534+0000 7fc4f7400700 1 -- 172.21.15.133:0/1267513447 <== osd.3 v2:172.21.15.133:6810/2675627022 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc500103a90 con 0x7fc4e40ac490 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] conn(0x7fc4e40ac490 msgr2=0x7fc4e40ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] conn(0x7fc4e40ac490 0x7fc4e40ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc4e40a2cb0 msgr2=0x7fc4e40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc4e40a2cb0 0x7fc4e40a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006010 tx=0x7fc4f000b260 comp rx=0 tx=0).stop 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 msgr2=0x7fc50019b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 --2- 172.21.15.133:0/1267513447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc500102f80 0x7fc50019b270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e970 tx=0x7fc4ec00ec80 comp rx=0 tx=0).stop 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc4ff400700 1 -- 172.21.15.133:0/1267513447 reap_dead start 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 shutdown_connections 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 >> 172.21.15.133:0/1267513447 conn(0x7fc5000794d0 msgr2=0x7fc500101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 shutdown_connections 2024-05-09T08:28:38.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:38.535+0000 7fc505c00700 1 -- 172.21.15.133:0/1267513447 wait complete. 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 -- 172.21.15.133:0/1298368051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a030 msgr2=0x7f615806a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 --2- 172.21.15.133:0/1298368051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a030 0x7f615806a490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61440098f0 tx=0x7f6144009c00 comp rx=0 tx=0).stop 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 -- 172.21.15.133:0/1298368051 shutdown_connections 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 --2- 172.21.15.133:0/1298368051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f615806a9d0 0x7f615810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 --2- 172.21.15.133:0/1298368051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61581083a0 0x7f6158108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 --2- 172.21.15.133:0/1298368051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a030 0x7f615806a490 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.826+0000 7f615d000700 1 -- 172.21.15.133:0/1298368051 >> 172.21.15.133:0/1298368051 conn(0x7f61580781d0 msgr2=0x7f61580785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.827+0000 7f615d000700 1 -- 172.21.15.133:0/1298368051 shutdown_connections 2024-05-09T08:28:39.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.827+0000 7f615d000700 1 -- 172.21.15.133:0/1298368051 wait complete. 2024-05-09T08:28:39.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.828+0000 7f615d000700 1 Processor -- start 2024-05-09T08:28:39.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.828+0000 7f615d000700 1 -- start start 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615806a030 0x7f61581012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 0x7f6158101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61581083a0 0x7f6158101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6158112650 con 0x7f61581083a0 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f61581124d0 con 0x7f615806a9d0 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f615d000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61581127d0 con 0x7f615806a030 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 0x7f6158101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615806a030 0x7f61581012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 0x7f6158101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:53784/0 (socket says 172.21.15.133:53784) 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 -- 172.21.15.133:0/2034824756 learned_addr learned my addr 172.21.15.133:0/2034824756 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:39.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6156800700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61581083a0 0x7f6158101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 -- 172.21.15.133:0/2034824756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615806a030 msgr2=0x7f61581012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f615806a030 0x7f61581012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 -- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61581083a0 msgr2=0x7f6158101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61581083a0 0x7f6158101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.829+0000 7f6155400700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61581a70d0 con 0x7f615806a9d0 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f6155400700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 0x7f6158101830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6144005950 tx=0x7f614400fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:39.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144005140 con 0x7f615806a9d0 2024-05-09T08:28:39.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614400b560 con 0x7f615806a9d0 2024-05-09T08:28:39.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144003ae0 con 0x7f615806a9d0 2024-05-09T08:28:39.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61581a7360 con 0x7f615806a9d0 2024-05-09T08:28:39.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.830+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a77b0 con 0x7f615806a9d0 2024-05-09T08:28:39.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.831+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6158062a30 con 0x7f615806a9d0 2024-05-09T08:28:39.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.833+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f614400ba80 con 0x7f615806a9d0 2024-05-09T08:28:39.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.834+0000 7f614ea00700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f613c0a2cb0 0x7f613c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f61440d0470 con 0x7f615806a9d0 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f6155e00700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f613c0a2cb0 0x7f613c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f614ea00700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] conn(0x7f613c0ac360 0x7f613c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6144001d30 con 0x7f613c0ac360 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f61440d0820 con 0x7f615806a9d0 2024-05-09T08:28:39.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.835+0000 7f6156800700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] conn(0x7f613c0ac360 0x7f613c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:39.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.836+0000 7f6155e00700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f613c0a2cb0 0x7f613c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6148009be0 tx=0x7f6148009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:39.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.836+0000 7f6156800700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] conn(0x7f613c0ac360 0x7f613c0ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:39.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.837+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== osd.7 v2:172.21.15.160:6816/3423087941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6144001d30 con 0x7f613c0ac360 2024-05-09T08:28:39.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.854+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 --> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6158062eb0 con 0x7f613c0ac360 2024-05-09T08:28:39.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.855+0000 7f614ea00700 1 -- 172.21.15.133:0/2034824756 <== osd.7 v2:172.21.15.160:6816/3423087941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6158062eb0 con 0x7f613c0ac360 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] conn(0x7f613c0ac360 msgr2=0x7f613c0ae780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] conn(0x7f613c0ac360 0x7f613c0ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f613c0a2cb0 msgr2=0x7f613c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f613c0a2cb0 0x7f613c0a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6148009be0 tx=0x7f6148009260 comp rx=0 tx=0).stop 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 msgr2=0x7f6158101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 --2- 172.21.15.133:0/2034824756 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f615806a9d0 0x7f6158101830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6144005950 tx=0x7f614400fb20 comp rx=0 tx=0).stop 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f6156800700 1 -- 172.21.15.133:0/2034824756 reap_dead start 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 shutdown_connections 2024-05-09T08:28:39.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 >> 172.21.15.133:0/2034824756 conn(0x7f61580781d0 msgr2=0x7f61580ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:39.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 shutdown_connections 2024-05-09T08:28:39.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.856+0000 7f615d000700 1 -- 172.21.15.133:0/2034824756 wait complete. 2024-05-09T08:28:39.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:39 smithi133 ceph-mon[29610]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:39.985 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:39 smithi188 ceph-mon[34645]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:40.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:39 smithi160 ceph-mon[34070]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:41.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 -- 172.21.15.133:0/3240431592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 msgr2=0x7f206810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 --2- 172.21.15.133:0/3240431592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206810aa10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b0d0 tx=0x7f205c00b3e0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 -- 172.21.15.133:0/3240431592 shutdown_connections 2024-05-09T08:28:41.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 --2- 172.21.15.133:0/3240431592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104f40 0x7f2068111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 --2- 172.21.15.133:0/3240431592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206810aa10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 --2- 172.21.15.133:0/3240431592 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104580 0x7f2068104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 -- 172.21.15.133:0/3240431592 >> 172.21.15.133:0/3240431592 conn(0x7f2068100050 msgr2=0x7f2068102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 -- 172.21.15.133:0/3240431592 shutdown_connections 2024-05-09T08:28:41.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.185+0000 7f2070000700 1 -- 172.21.15.133:0/3240431592 wait complete. 2024-05-09T08:28:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.186+0000 7f2070000700 1 Processor -- start 2024-05-09T08:28:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.186+0000 7f2070000700 1 -- start start 2024-05-09T08:28:41.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104580 0x7f20681a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104f40 0x7f20681a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20681148f0 con 0x7f206810a630 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2068114770 con 0x7f2068104f40 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f2070000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2068114a70 con 0x7f2068104580 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f206cc00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104f40 0x7f20681a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f206d600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104580 0x7f20681a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:41.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f206e000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f206cc00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104f40 0x7f20681a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:53798/0 (socket says 172.21.15.133:53798) 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.187+0000 7f206cc00700 1 -- 172.21.15.133:0/1744757260 learned_addr learned my addr 172.21.15.133:0/1744757260 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 -- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104f40 msgr2=0x7f20681a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2068104f40 0x7f20681a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 -- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 msgr2=0x7f206819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206819b230 con 0x7f2068104580 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206e000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206810a630 0x7f206819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f206d600700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104580 0x7f20681a0910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f80 tx=0x7f205c007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c003c00 con 0x7f2068104580 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f205c022070 con 0x7f2068104580 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206819b4c0 con 0x7f2068104580 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c01b6c0 con 0x7f2068104580 2024-05-09T08:28:41.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.188+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20681a77e0 con 0x7f2068104580 2024-05-09T08:28:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.189+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2068062ab0 con 0x7f2068104580 2024-05-09T08:28:41.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.191+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f205c01d470 con 0x7f2068104580 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f2066000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20500a2d30 0x7f20500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f205c00f020 con 0x7f2068104580 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f206cc00700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20500a2d30 0x7f20500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f2066000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] conn(0x7f20500ac510 0x7f20500ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f205c0020b0 con 0x7f20500ac510 2024-05-09T08:28:41.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f205c0d05f0 con 0x7f2068104580 2024-05-09T08:28:41.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.193+0000 7f206e000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] conn(0x7f20500ac510 0x7f20500ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:41.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.194+0000 7f206cc00700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20500a2d30 0x7f20500a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2054009680 tx=0x7f2054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:41.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.194+0000 7f206e000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] conn(0x7f20500ac510 0x7f20500ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:41.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.195+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== osd.1 v2:172.21.15.160:6800/2987709381 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f205c0020b0 con 0x7f20500ac510 2024-05-09T08:28:41.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.213+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 --> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2068062eb0 con 0x7f20500ac510 2024-05-09T08:28:41.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.213+0000 7f2066000700 1 -- 172.21.15.133:0/1744757260 <== osd.1 v2:172.21.15.160:6800/2987709381 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2068062eb0 con 0x7f20500ac510 2024-05-09T08:28:41.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] conn(0x7f20500ac510 msgr2=0x7f20500ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] conn(0x7f20500ac510 0x7f20500ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20500a2d30 msgr2=0x7f20500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20500a2d30 0x7f20500a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2054009680 tx=0x7f2054009260 comp rx=0 tx=0).stop 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104580 msgr2=0x7f20681a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 --2- 172.21.15.133:0/1744757260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068104580 0x7f20681a0910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f205c000f80 tx=0x7f205c007a20 comp rx=0 tx=0).stop 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f206e000700 1 -- 172.21.15.133:0/1744757260 reap_dead start 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.214+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 shutdown_connections 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.215+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 >> 172.21.15.133:0/1744757260 conn(0x7f2068100050 msgr2=0x7f20681015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.215+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 shutdown_connections 2024-05-09T08:28:41.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.215+0000 7f2070000700 1 -- 172.21.15.133:0/1744757260 wait complete. 2024-05-09T08:28:41.657 INFO:teuthology.orchestra.run.smithi133.stdout:111669149763 2024-05-09T08:28:41.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T08:28:42.133 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:41 smithi133 ceph-mon[29610]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:42.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:41 smithi160 ceph-mon[34070]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:42.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:41 smithi188 ceph-mon[34645]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 -- 172.21.15.133:0/2489571229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0107660 msgr2=0x7fc0f010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 --2- 172.21.15.133:0/2489571229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0107660 0x7fc0f010f230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009990 tx=0x7fc0e8009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 -- 172.21.15.133:0/2489571229 shutdown_connections 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 --2- 172.21.15.133:0/2489571229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0107660 0x7fc0f010f230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 --2- 172.21.15.133:0/2489571229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f01025d0 0x7fc0f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 --2- 172.21.15.133:0/2489571229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f0102f80 0x7fc0f0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.294+0000 7fc0f7400700 1 -- 172.21.15.133:0/2489571229 >> 172.21.15.133:0/2489571229 conn(0x7fc0f0079480 msgr2=0x7fc0f0079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:42.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.295+0000 7fc0f7400700 1 -- 172.21.15.133:0/2489571229 shutdown_connections 2024-05-09T08:28:42.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.295+0000 7fc0f7400700 1 -- 172.21.15.133:0/2489571229 wait complete. 2024-05-09T08:28:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.296+0000 7fc0f7400700 1 Processor -- start 2024-05-09T08:28:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.296+0000 7fc0f7400700 1 -- start start 2024-05-09T08:28:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 0x7fc0f01a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:42.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0102f80 0x7fc0f01a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f0107660 0x7fc0f019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0112690 con 0x7fc0f0107660 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0112510 con 0x7fc0f01025d0 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f7400700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f0112810 con 0x7fc0f0102f80 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f5400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f0107660 0x7fc0f019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f4a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 0x7fc0f01a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f4a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 0x7fc0f01a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:53838/0 (socket says 172.21.15.133:53838) 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f5400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f0107660 0x7fc0f019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40604/0 (socket says 172.21.15.133:40604) 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0f4a00700 1 -- 172.21.15.133:0/3277800282 learned_addr learned my addr 172.21.15.133:0/3277800282 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.297+0000 7fc0efe00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0102f80 0x7fc0f01a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 -- 172.21.15.133:0/3277800282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0102f80 msgr2=0x7fc0f01a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f0102f80 0x7fc0f01a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 -- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f0107660 msgr2=0x7fc0f019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0f0107660 0x7fc0f019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f019b250 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f4a00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 0x7fc0f01a0930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400ef10 tx=0x7fc0e400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400c9c0 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e4019070 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f019b4e0 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007e60 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.298+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f01a7800 con 0x7fc0f01025d0 2024-05-09T08:28:42.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.299+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc0f00758b0 con 0x7fc0f01025d0 2024-05-09T08:28:42.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.302+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc0e4012050 con 0x7fc0f01025d0 2024-05-09T08:28:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.303+0000 7fc0ed600700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0d80a2d30 0x7fc0d80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.303+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc0e40cf630 con 0x7fc0f01025d0 2024-05-09T08:28:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.303+0000 7fc0efe00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0d80a2d30 0x7fc0d80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0ed600700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] conn(0x7fc0d80ac510 0x7fc0d80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0e4026ea0 con 0x7fc0d80ac510 2024-05-09T08:28:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc0e40cf9e0 con 0x7fc0f01025d0 2024-05-09T08:28:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0f5400700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] conn(0x7fc0d80ac510 0x7fc0d80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0efe00700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0d80a2d30 0x7fc0d80a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f010f5e0 tx=0x7fc0dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.304+0000 7fc0f5400700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] conn(0x7fc0d80ac510 0x7fc0d80ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:42.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.305+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== osd.6 v2:172.21.15.133:6818/1108533027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0e4026ea0 con 0x7fc0d80ac510 2024-05-09T08:28:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.323+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 --> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc0f0103a60 con 0x7fc0d80ac510 2024-05-09T08:28:42.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.324+0000 7fc0ed600700 1 -- 172.21.15.133:0/3277800282 <== osd.6 v2:172.21.15.133:6818/1108533027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc0f0103a60 con 0x7fc0d80ac510 2024-05-09T08:28:42.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.324+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] conn(0x7fc0d80ac510 msgr2=0x7fc0d80ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.324+0000 7fc0f7400700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] conn(0x7fc0d80ac510 0x7fc0d80ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0d80a2d30 msgr2=0x7fc0d80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0d80a2d30 0x7fc0d80a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0f010f5e0 tx=0x7fc0dc009260 comp rx=0 tx=0).stop 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 msgr2=0x7fc0f01a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 --2- 172.21.15.133:0/3277800282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0f01025d0 0x7fc0f01a0930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400ef10 tx=0x7fc0e400c3a0 comp rx=0 tx=0).stop 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f5400700 1 -- 172.21.15.133:0/3277800282 reap_dead start 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 shutdown_connections 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 >> 172.21.15.133:0/3277800282 conn(0x7fc0f0079480 msgr2=0x7fc0f01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 shutdown_connections 2024-05-09T08:28:42.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:42.325+0000 7fc0f7400700 1 -- 172.21.15.133:0/3277800282 wait complete. 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 -- 172.21.15.133:0/849219450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22201025d0 msgr2=0x7f22201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 --2- 172.21.15.133:0/849219450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22201025d0 0x7f22201029b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22100098f0 tx=0x7f2210009c00 comp rx=0 tx=0).stop 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 -- 172.21.15.133:0/849219450 shutdown_connections 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 --2- 172.21.15.133:0/849219450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2220107660 0x7f222010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 --2- 172.21.15.133:0/849219450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22201025d0 0x7f22201029b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 --2- 172.21.15.133:0/849219450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2220102f80 0x7f2220106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 -- 172.21.15.133:0/849219450 >> 172.21.15.133:0/849219450 conn(0x7f2220079480 msgr2=0x7f2220079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:43.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 -- 172.21.15.133:0/849219450 shutdown_connections 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.851+0000 7f2225e00700 1 -- 172.21.15.133:0/849219450 wait complete. 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.852+0000 7f2225e00700 1 Processor -- start 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.852+0000 7f2225e00700 1 -- start start 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22201025d0 0x7f222019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 0x7f222019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2220107660 0x7f222019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22201126a0 con 0x7f2220107660 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2220112520 con 0x7f22201025d0 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f2225e00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2220112820 con 0x7f2220102f80 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 0x7f222019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 0x7f222019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:46218/0 (socket says 172.21.15.133:46218) 2024-05-09T08:28:43.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 -- 172.21.15.133:0/4025330214 learned_addr learned my addr 172.21.15.133:0/4025330214 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221ea00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22201025d0 0x7f222019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221f400700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2220107660 0x7f222019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 -- 172.21.15.133:0/4025330214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22201025d0 msgr2=0x7f222019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22201025d0 0x7f222019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.853+0000 7f221e000700 1 -- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2220107660 msgr2=0x7f222019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f221e000700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2220107660 0x7f222019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f221e000700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222019fd80 con 0x7f2220102f80 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f221ea00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22201025d0 0x7f222019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f221e000700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 0x7f222019b300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f220c00ec70 tx=0x7f220c00ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00cc90 con 0x7f2220102f80 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c00ce30 con 0x7f2220102f80 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c010470 con 0x7f2220102f80 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201a0070 con 0x7f2220102f80 2024-05-09T08:28:43.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.854+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a78f0 con 0x7f2220102f80 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.856+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2220100e20 con 0x7f2220102f80 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.857+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f220c004000 con 0x7f2220102f80 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.859+0000 7f2224a00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22040a2d30 0x7f22040a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.859+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f220c0cf3f0 con 0x7f2220102f80 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.859+0000 7f221ea00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22040a2d30 0x7f22040a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.860+0000 7f2224a00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] conn(0x7f22040ac510 0x7f22040ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.860+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22040aefe0 con 0x7f22040ac510 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.860+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f220c0d6050 con 0x7f2220102f80 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.860+0000 7f221f400700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] conn(0x7f22040ac510 0x7f22040ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.860+0000 7f221ea00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22040a2d30 0x7f22040a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2210009810 tx=0x7f2210019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.861+0000 7f221f400700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] conn(0x7f22040ac510 0x7f22040ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.862+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== osd.5 v2:172.21.15.188:6808/485185361 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22040aefe0 con 0x7f22040ac510 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.881+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 --> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2220103a60 con 0x7f22040ac510 2024-05-09T08:28:43.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2224a00700 1 -- 172.21.15.133:0/4025330214 <== osd.5 v2:172.21.15.188:6808/485185361 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2220103a60 con 0x7f22040ac510 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] conn(0x7f22040ac510 msgr2=0x7f22040ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] conn(0x7f22040ac510 0x7f22040ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22040a2d30 msgr2=0x7f22040a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22040a2d30 0x7f22040a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2210009810 tx=0x7f2210019040 comp rx=0 tx=0).stop 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 msgr2=0x7f222019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.882+0000 7f2225e00700 1 --2- 172.21.15.133:0/4025330214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2220102f80 0x7f222019b300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f220c00ec70 tx=0x7f220c00ef80 comp rx=0 tx=0).stop 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.883+0000 7f221f400700 1 -- 172.21.15.133:0/4025330214 reap_dead start 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.883+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 shutdown_connections 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.883+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 >> 172.21.15.133:0/4025330214 conn(0x7f2220079480 msgr2=0x7f2220100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:43.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.883+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 shutdown_connections 2024-05-09T08:28:43.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:41.883+0000 7f2225e00700 1 -- 172.21.15.133:0/4025330214 wait complete. 2024-05-09T08:28:43.966 INFO:teuthology.orchestra.run.smithi133.stdout:167503724602 2024-05-09T08:28:43.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T08:28:44.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:43 smithi133 ceph-mon[29610]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 -- 172.21.15.133:0/701725193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e41023d0 msgr2=0x7f63e4102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 --2- 172.21.15.133:0/701725193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e41023d0 0x7f63e4102850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f63d00099c0 tx=0x7f63d0009cd0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 -- 172.21.15.133:0/701725193 shutdown_connections 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 --2- 172.21.15.133:0/701725193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e4102d90 0x7f63e410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 --2- 172.21.15.133:0/701725193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63e41083e0 0x7f63e41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 --2- 172.21.15.133:0/701725193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e41023d0 0x7f63e4102850 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 -- 172.21.15.133:0/701725193 >> 172.21.15.133:0/701725193 conn(0x7f63e4079480 msgr2=0x7f63e4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 -- 172.21.15.133:0/701725193 shutdown_connections 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.721+0000 7f63ec600700 1 -- 172.21.15.133:0/701725193 wait complete. 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.722+0000 7f63ec600700 1 Processor -- start 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.722+0000 7f63ec600700 1 -- start start 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 0x7f63e419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e4102d90 0x7f63e419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63e41083e0 0x7f63e4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63e4112660 con 0x7f63e4102d90 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41124e0 con 0x7f63e41083e0 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ec600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41127e0 con 0x7f63e41023d0 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63e9200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e4102d90 0x7f63e419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63e9c00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 0x7f63e419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:44.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63e9c00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 0x7f63e419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:46160/0 (socket says 172.21.15.133:46160) 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63e9c00700 1 -- 172.21.15.133:0/3755280992 learned_addr learned my addr 172.21.15.133:0/3755280992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.723+0000 7f63ea600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63e41083e0 0x7f63e4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 -- 172.21.15.133:0/3755280992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63e41083e0 msgr2=0x7f63e4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63e41083e0 0x7f63e4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 -- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e4102d90 msgr2=0x7f63e419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63e4102d90 0x7f63e419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e4199080 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e9c00700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 0x7f63e419e710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e9a0 tx=0x7f63d800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d800cbd0 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e4199370 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d800cd70 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8018750 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.724+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a55e0 con 0x7f63e41023d0 2024-05-09T08:28:44.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.726+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63e4062ab0 con 0x7f63e41023d0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.728+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f63d8010430 con 0x7f63e41023d0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.730+0000 7f63e2000700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63cc0a2d30 0x7f63cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.730+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f63d80cfac0 con 0x7f63e41023d0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.730+0000 7f63e9200700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63cc0a2d30 0x7f63cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.730+0000 7f63e2000700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] conn(0x7f63cc0ac510 0x7f63cc0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.730+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63cc0aefe0 con 0x7f63cc0ac510 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.731+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f63d80cfe70 con 0x7f63e41023d0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.731+0000 7f63ea600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] conn(0x7f63cc0ac510 0x7f63cc0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.731+0000 7f63e9200700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63cc0a2d30 0x7f63cc0a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63e4079aa0 tx=0x7f63d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.732+0000 7f63ea600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] conn(0x7f63cc0ac510 0x7f63cc0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.733+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== osd.11 v2:172.21.15.188:6824/2170090955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63cc0aefe0 con 0x7f63cc0ac510 2024-05-09T08:28:44.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.756+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 --> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63e4062eb0 con 0x7f63cc0ac510 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.757+0000 7f63e2000700 1 -- 172.21.15.133:0/3755280992 <== osd.11 v2:172.21.15.188:6824/2170090955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f63e4062eb0 con 0x7f63cc0ac510 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] conn(0x7f63cc0ac510 msgr2=0x7f63cc0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] conn(0x7f63cc0ac510 0x7f63cc0ae930 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63cc0a2d30 msgr2=0x7f63cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63cc0a2d30 0x7f63cc0a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63e4079aa0 tx=0x7f63d0005fb0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 msgr2=0x7f63e419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 --2- 172.21.15.133:0/3755280992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63e41023d0 0x7f63e419e710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e9a0 tx=0x7f63d800ecb0 comp rx=0 tx=0).stop 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ea600700 1 -- 172.21.15.133:0/3755280992 reap_dead start 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 shutdown_connections 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 >> 172.21.15.133:0/3755280992 conn(0x7f63e4079480 msgr2=0x7f63e4100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 shutdown_connections 2024-05-09T08:28:44.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:39.758+0000 7f63ec600700 1 -- 172.21.15.133:0/3755280992 wait complete. 2024-05-09T08:28:44.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:43 smithi160 ceph-mon[34070]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-09T08:28:44.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:43 smithi188 ceph-mon[34645]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-09T08:28:45.110 INFO:teuthology.orchestra.run.smithi133.stdout:347892351000 2024-05-09T08:28:45.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 -- 172.21.15.133:0/186013755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 msgr2=0x7f204c1158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 --2- 172.21.15.133:0/186013755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 0x7f204c1158e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f204400b0a0 tx=0x7f204400b3b0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 -- 172.21.15.133:0/186013755 shutdown_connections 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 --2- 172.21.15.133:0/186013755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 0x7f204c1158e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 --2- 172.21.15.133:0/186013755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204c074550 0x7f204c074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 --2- 172.21.15.133:0/186013755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074e70 0x7f204c112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 -- 172.21.15.133:0/186013755 >> 172.21.15.133:0/186013755 conn(0x7f204c0794d0 msgr2=0x7f204c0798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 -- 172.21.15.133:0/186013755 shutdown_connections 2024-05-09T08:28:45.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.326+0000 7f2052000700 1 -- 172.21.15.133:0/186013755 wait complete. 2024-05-09T08:28:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 Processor -- start 2024-05-09T08:28:45.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 -- start start 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 0x7f204c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204c074e70 0x7f204c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 0x7f204c1a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f204c118c70 con 0x7f204c1134f0 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f204c118af0 con 0x7f204c074e70 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.328+0000 7f2052000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f204c118df0 con 0x7f204c074550 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 0x7f204c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204aa00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204c074e70 0x7f204c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 0x7f204c1a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 0x7f204c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:42640/0 (socket says 172.21.15.133:42640) 2024-05-09T08:28:45.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 -- 172.21.15.133:0/658159948 learned_addr learned my addr 172.21.15.133:0/658159948 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 -- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204c074e70 msgr2=0x7f204c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204c074e70 0x7f204c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 -- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 msgr2=0x7f204c1a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204c1134f0 0x7f204c1a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1a40d0 con 0x7f204c074550 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.329+0000 7f204b400700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 0x7f204c19f0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9a0 tx=0x7f203c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:45.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.330+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00ca10 con 0x7f204c074550 2024-05-09T08:28:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.330+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c1a43c0 con 0x7f204c074550 2024-05-09T08:28:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.330+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f203c00cbb0 con 0x7f204c074550 2024-05-09T08:28:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.330+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c018680 con 0x7f204c074550 2024-05-09T08:28:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.330+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1abc40 con 0x7f204c074550 2024-05-09T08:28:45.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.331+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f204c112cb0 con 0x7f204c074550 2024-05-09T08:28:45.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.334+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f203c010430 con 0x7f204c074550 2024-05-09T08:28:45.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f2043e00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20300a2cb0 0x7f20300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:45.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f203c0cf260 con 0x7f204c074550 2024-05-09T08:28:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f204aa00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20300a2cb0 0x7f20300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:45.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f2043e00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] conn(0x7f20300ac490 0x7f20300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f203c015e80 con 0x7f20300ac490 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.336+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f203c0d6050 con 0x7f204c074550 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.337+0000 7f204be00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] conn(0x7f20300ac490 0x7f20300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.337+0000 7f204aa00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20300a2cb0 0x7f20300a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f204c10fe10 tx=0x7f2038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.337+0000 7f204be00700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] conn(0x7f20300ac490 0x7f20300ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:45.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.338+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== osd.10 v2:172.21.15.160:6824/993185215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f203c015e80 con 0x7f20300ac490 2024-05-09T08:28:45.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.357+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 --> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f204c075c70 con 0x7f20300ac490 2024-05-09T08:28:45.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.357+0000 7f2043e00700 1 -- 172.21.15.133:0/658159948 <== osd.10 v2:172.21.15.160:6824/993185215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f204c075c70 con 0x7f20300ac490 2024-05-09T08:28:45.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] conn(0x7f20300ac490 msgr2=0x7f20300ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] conn(0x7f20300ac490 0x7f20300ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20300a2cb0 msgr2=0x7f20300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f20300a2cb0 0x7f20300a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f204c10fe10 tx=0x7f2038009260 comp rx=0 tx=0).stop 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 msgr2=0x7f204c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 --2- 172.21.15.133:0/658159948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f204c074550 0x7f204c19f0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f203c00e9a0 tx=0x7f203c00ecb0 comp rx=0 tx=0).stop 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f204be00700 1 -- 172.21.15.133:0/658159948 reap_dead start 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 shutdown_connections 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 >> 172.21.15.133:0/658159948 conn(0x7f204c0794d0 msgr2=0x7f204c112250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 shutdown_connections 2024-05-09T08:28:45.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:45.358+0000 7f2052000700 1 -- 172.21.15.133:0/658159948 wait complete. 2024-05-09T08:28:45.986 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:45 smithi133 ceph-mon[29610]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:46.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:45 smithi160 ceph-mon[34070]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:46.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:45 smithi188 ceph-mon[34645]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.441+0000 7f3941c00700 1 -- 172.21.15.133:0/676840566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c10ed80 msgr2=0x7f393c111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 --2- 172.21.15.133:0/676840566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c10ed80 0x7f393c111170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3934009f90 tx=0x7f393400b330 comp rx=0 tx=0).stop 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 -- 172.21.15.133:0/676840566 shutdown_connections 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 --2- 172.21.15.133:0/676840566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c10ed80 0x7f393c111170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 --2- 172.21.15.133:0/676840566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393c074550 0x7f393c074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 --2- 172.21.15.133:0/676840566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074e70 0x7f393c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 -- 172.21.15.133:0/676840566 >> 172.21.15.133:0/676840566 conn(0x7f393c079480 msgr2=0x7f393c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:46.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 -- 172.21.15.133:0/676840566 shutdown_connections 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.442+0000 7f3941c00700 1 -- 172.21.15.133:0/676840566 wait complete. 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 Processor -- start 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 -- start start 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 0x7f393c1a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c074e70 0x7f393c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393c10ed80 0x7f393c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1148a0 con 0x7f393c074550 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f393c114720 con 0x7f393c10ed80 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.443+0000 7f3941c00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f393c114a20 con 0x7f393c074e70 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 0x7f393c1a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:46.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 0x7f393c1a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39042/0 (socket says 172.21.15.133:39042) 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 -- 172.21.15.133:0/266780750 learned_addr learned my addr 172.21.15.133:0/266780750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f3933e00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c074e70 0x7f393c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393b400700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393c10ed80 0x7f393c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 -- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c074e70 msgr2=0x7f393c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f393c074e70 0x7f393c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 -- 172.21.15.133:0/266780750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393c10ed80 msgr2=0x7f393c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393c10ed80 0x7f393c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c19b140 con 0x7f393c074550 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.444+0000 7f393aa00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 0x7f393c1a0820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f392c00d980 tx=0x7f392c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.445+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c016070 con 0x7f393c074550 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.445+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c19b430 con 0x7f393c074550 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.445+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c004d10 con 0x7f393c074550 2024-05-09T08:28:46.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.445+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c005670 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.445+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c1a76f0 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.446+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f393c0fff10 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.448+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f392c004750 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.449+0000 7f3933400700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f39200a2cb0 0x7f39200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.449+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f392c0cf5f0 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.449+0000 7f3933e00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f39200a2cb0 0x7f39200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f3933400700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] conn(0x7f39200ac490 0x7f39200ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f392c015ec0 con 0x7f39200ac490 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f392c0cf9a0 con 0x7f393c074550 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f393b400700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] conn(0x7f39200ac490 0x7f39200ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:46.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f3933e00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f39200a2cb0 0x7f39200a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3924009750 tx=0x7f3924009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.450+0000 7f393b400700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] conn(0x7f39200ac490 0x7f39200ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.451+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== osd.8 v2:172.21.15.188:6816/4036931105 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f392c015ec0 con 0x7f39200ac490 2024-05-09T08:28:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.469+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 --> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f393c075c30 con 0x7f39200ac490 2024-05-09T08:28:46.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.470+0000 7f3933400700 1 -- 172.21.15.133:0/266780750 <== osd.8 v2:172.21.15.188:6816/4036931105 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f393c075c30 con 0x7f39200ac490 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] conn(0x7f39200ac490 msgr2=0x7f39200ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] conn(0x7f39200ac490 0x7f39200ae8b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f39200a2cb0 msgr2=0x7f39200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f39200a2cb0 0x7f39200a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3924009750 tx=0x7f3924009310 comp rx=0 tx=0).stop 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 msgr2=0x7f393c1a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 --2- 172.21.15.133:0/266780750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f393c074550 0x7f393c1a0820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f392c00d980 tx=0x7f392c00dc90 comp rx=0 tx=0).stop 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f393b400700 1 -- 172.21.15.133:0/266780750 reap_dead start 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 shutdown_connections 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 >> 172.21.15.133:0/266780750 conn(0x7f393c079480 msgr2=0x7f393c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:46.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 shutdown_connections 2024-05-09T08:28:46.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:46.471+0000 7f3941c00700 1 -- 172.21.15.133:0/266780750 wait complete. 2024-05-09T08:28:46.864 INFO:teuthology.orchestra.run.smithi133.stdout:292057776161 2024-05-09T08:28:46.865 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T08:28:47.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:46 smithi160 ceph-mon[34070]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:47.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:46 smithi133 ceph-mon[29610]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:46 smithi188 ceph-mon[34645]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:47.797 INFO:teuthology.orchestra.run.smithi133.stdout:201863462964 2024-05-09T08:28:47.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T08:28:48.257 INFO:teuthology.orchestra.run.smithi133.stdout:55834574928 2024-05-09T08:28:48.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T08:28:48.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 -- 172.21.15.133:0/1306368946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 msgr2=0x7f67c010e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 --2- 172.21.15.133:0/1306368946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c010e5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009990 tx=0x7f67ac009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 -- 172.21.15.133:0/1306368946 shutdown_connections 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 --2- 172.21.15.133:0/1306368946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c0110ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 --2- 172.21.15.133:0/1306368946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00746f0 0x7f67c0074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 --2- 172.21.15.133:0/1306368946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c010e5a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 -- 172.21.15.133:0/1306368946 >> 172.21.15.133:0/1306368946 conn(0x7f67c00fdd30 msgr2=0x7f67c0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 -- 172.21.15.133:0/1306368946 shutdown_connections 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.599+0000 7f67bf200700 1 -- 172.21.15.133:0/1306368946 wait complete. 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.600+0000 7f67bf200700 1 Processor -- start 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.600+0000 7f67bf200700 1 -- start start 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00746f0 0x7f67c01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:48.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0114780 con 0x7f67c0075010 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0114600 con 0x7f67c010eae0 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bf200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67c0114900 con 0x7f67c00746f0 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bd400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39280/0 (socket says 172.21.15.133:39280) 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 -- 172.21.15.133:0/2225345865 learned_addr learned my addr 172.21.15.133:0/2225345865 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bd400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c01a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39056/0 (socket says 172.21.15.133:39056) 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67bde00700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00746f0 0x7f67c01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 -- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00746f0 msgr2=0x7f67c01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c00746f0 0x7f67c01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 -- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 msgr2=0x7f67c01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67c0075010 0x7f67c01a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.601+0000 7f67be800700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c019b1c0 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67be800700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c019aa60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67b800ef90 tx=0x7f67b800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b80092e0 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b8004bb0 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c019b450 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004280 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.602+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c01a7720 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.603+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67c0062a30 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.605+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f67b801b030 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.606+0000 7f67b6a00700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f67a40a2d30 0x7f67a40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.606+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f67b80cf530 con 0x7f67c010eae0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.606+0000 7f67bde00700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f67a40a2d30 0x7f67a40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.607+0000 7f67b6a00700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] conn(0x7f67a40ac510 0x7f67a40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:48.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.607+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67b801fe50 con 0x7f67a40ac510 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.607+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f67b80cf8e0 con 0x7f67c010eae0 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.607+0000 7f67bde00700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f67a40a2d30 0x7f67a40a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f67c0111350 tx=0x7f67b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.607+0000 7f67bd400700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] conn(0x7f67a40ac510 0x7f67a40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.608+0000 7f67bd400700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] conn(0x7f67a40ac510 0x7f67a40ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.609+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== osd.2 v2:172.21.15.188:6800/2504154225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67b801fe50 con 0x7f67a40ac510 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.626+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 --> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67c0062eb0 con 0x7f67a40ac510 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.627+0000 7f67b6a00700 1 -- 172.21.15.133:0/2225345865 <== osd.2 v2:172.21.15.188:6800/2504154225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f67c0062eb0 con 0x7f67a40ac510 2024-05-09T08:28:48.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.627+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] conn(0x7f67a40ac510 msgr2=0x7f67a40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.627+0000 7f67bf200700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] conn(0x7f67a40ac510 0x7f67a40ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.628+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f67a40a2d30 msgr2=0x7f67a40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.628+0000 7f67bf200700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f67a40a2d30 0x7f67a40a51f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f67c0111350 tx=0x7f67b0009260 comp rx=0 tx=0).stop 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.628+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 msgr2=0x7f67c019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.628+0000 7f67bf200700 1 --2- 172.21.15.133:0/2225345865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67c010eae0 0x7f67c019aa60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67b800ef90 tx=0x7f67b800c3a0 comp rx=0 tx=0).stop 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.628+0000 7f67be800700 1 -- 172.21.15.133:0/2225345865 reap_dead start 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.629+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 shutdown_connections 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.629+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 >> 172.21.15.133:0/2225345865 conn(0x7f67c00fdd30 msgr2=0x7f67c00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.629+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 shutdown_connections 2024-05-09T08:28:48.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:47.629+0000 7f67bf200700 1 -- 172.21.15.133:0/2225345865 wait complete. 2024-05-09T08:28:49.512 INFO:teuthology.orchestra.run.smithi133.stdout:257698037803 2024-05-09T08:28:49.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T08:28:49.881 INFO:teuthology.orchestra.run.smithi133.stdout:223338299438 2024-05-09T08:28:49.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T08:28:50.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:49 smithi160 ceph-mon[34070]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:50.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:49 smithi133 ceph-mon[29610]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:50.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:49 smithi188 ceph-mon[34645]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:28:50.231 INFO:teuthology.orchestra.run.smithi133.stdout:77309411404 2024-05-09T08:28:50.231 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T08:28:50.328 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:50.386 INFO:teuthology.orchestra.run.smithi133.stdout:313532612636 2024-05-09T08:28:50.387 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T08:28:50.480 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.047 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.054 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.060 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.899 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:28:51.911 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:51 smithi133 ceph-mon[29610]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:52.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:51 smithi160 ceph-mon[34070]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:52.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:51 smithi188 ceph-mon[34645]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:28:53.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:53 smithi133 ceph-mon[29610]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:54.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:53 smithi160 ceph-mon[34070]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:53 smithi188 ceph-mon[34645]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:54.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:54 smithi133 ceph-mon[29610]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T08:28:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:54 smithi160 ceph-mon[34070]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T08:28:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:54 smithi188 ceph-mon[34645]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T08:28:56.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:55 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:28:56.174 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:55 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:28:56.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:55 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:28:57.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T08:28:57.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-09T08:28:57.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T08:28:57.134 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-05-09T08:28:57.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T08:28:57.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:56 smithi133 ceph-mon[29610]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T08:28:57.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:56 smithi160 ceph-mon[34070]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T08:28:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:56 smithi188 ceph-mon[34645]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T08:28:58.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T08:28:58.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T08:28:58.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.018 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:57 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 -- 172.21.15.133:0/3603579759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc81083a0 msgr2=0x7fdfc8108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/3603579759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc81083a0 0x7fdfc8108780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009990 tx=0x7fdfbc009ca0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 -- 172.21.15.133:0/3603579759 shutdown_connections 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/3603579759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a9d0 0x7fdfc810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/3603579759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc81083a0 0x7fdfc8108780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/3603579759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc806a030 0x7fdfc806a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 -- 172.21.15.133:0/3603579759 >> 172.21.15.133:0/3603579759 conn(0x7fdfc80781d0 msgr2=0x7fdfc80785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 -- 172.21.15.133:0/3603579759 shutdown_connections 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.071+0000 7fdfcfa00700 1 -- 172.21.15.133:0/3603579759 wait complete. 2024-05-09T08:28:58.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.072+0000 7fdfcfa00700 1 Processor -- start 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.072+0000 7fdfcfa00700 1 -- start start 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 0x7fdfc819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112650 con 0x7fdfc806a030 2024-05-09T08:28:58.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc81124d0 con 0x7fdfc806a9d0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcfa00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc81127d0 con 0x7fdfc81083a0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfc7e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcda00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfc7e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:45840/0 (socket says 172.21.15.133:45840) 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfcda00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:49400/0 (socket says 172.21.15.133:49400) 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.073+0000 7fdfc7e00700 1 -- 172.21.15.133:0/1166162315 learned_addr learned my addr 172.21.15.133:0/1166162315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcd000700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 0x7fdfc819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 -- 172.21.15.133:0/1166162315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 msgr2=0x7fdfc819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 -- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 msgr2=0x7fdfc819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 0x7fdfc819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 -- 172.21.15.133:0/1166162315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc8199130 con 0x7fdfc81083a0 2024-05-09T08:28:58.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfc7e00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcd000700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 0x7fdfc819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcda00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000efb0 tx=0x7fdfc000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000ca50 con 0x7fdfc81083a0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc000cbf0 con 0x7fdfc81083a0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc8199420 con 0x7fdfc81083a0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.074+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00048b0 con 0x7fdfc81083a0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.075+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a5690 con 0x7fdfc81083a0 2024-05-09T08:28:58.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.076+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8069010 con 0x7fdfc81083a0 2024-05-09T08:28:58.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.078+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdfc0012050 con 0x7fdfc81083a0 2024-05-09T08:28:58.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.079+0000 7fdfc5600700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 0x7fdfb00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:58.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.080+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdfc00cead0 con 0x7fdfc81083a0 2024-05-09T08:28:58.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.080+0000 7fdfcd000700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 0x7fdfb00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:58.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.080+0000 7fdfcd000700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 0x7fdfb00a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004820 tx=0x7fdfbc004690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:58.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.083+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfc0022060 con 0x7fdfc81083a0 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:57 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 6]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:57 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:28:58.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.323+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdfc802db80 con 0x7fdfc81083a0 2024-05-09T08:28:58.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.323+0000 7fdfc5600700 1 -- 172.21.15.133:0/1166162315 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdfc0010040 con 0x7fdfc81083a0 2024-05-09T08:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout:55834574931 2024-05-09T08:28:58.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 msgr2=0x7fdfb00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:58.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 0x7fdfb00a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004820 tx=0x7fdfbc004690 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 msgr2=0x7fdfc81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000efb0 tx=0x7fdfc000c3a0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 shutdown_connections 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fdfb00a2d30 0x7fdfb00a51f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc81083a0 0x7fdfc81989d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc806a030 0x7fdfc819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 --2- 172.21.15.133:0/1166162315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfc806a9d0 0x7fdfc819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.326+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 >> 172.21.15.133:0/1166162315 conn(0x7fdfc80781d0 msgr2=0x7fdfc80fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.327+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 shutdown_connections 2024-05-09T08:28:58.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.327+0000 7fdfcfa00700 1 -- 172.21.15.133:0/1166162315 wait complete. 2024-05-09T08:28:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:58 smithi160 ceph-mon[34070]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T08:28:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:58 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1166162315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T08:28:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:58 smithi160 ceph-mon[34070]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T08:28:59.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:28:58 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1606180664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T08:28:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:58 smithi133 ceph-mon[29610]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T08:28:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:58 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1166162315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T08:28:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:58 smithi133 ceph-mon[29610]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T08:28:59.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:28:58 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1606180664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T08:28:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:58 smithi188 ceph-mon[34645]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T08:28:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:58 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1166162315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T08:28:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:58 smithi188 ceph-mon[34645]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T08:28:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:28:58 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1606180664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T08:28:59.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574931 for osd.1 2024-05-09T08:28:59.245 DEBUG:teuthology.parallel:result is None 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 -- 172.21.15.133:0/893783778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104f20 msgr2=0x7f63d81113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 --2- 172.21.15.133:0/893783778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104f20 0x7f63d81113e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b0d0 tx=0x7f63d000b3e0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 -- 172.21.15.133:0/893783778 shutdown_connections 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 --2- 172.21.15.133:0/893783778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104f20 0x7f63d81113e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 --2- 172.21.15.133:0/893783778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d810a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 --2- 172.21.15.133:0/893783778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104560 0x7f63d81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 -- 172.21.15.133:0/893783778 >> 172.21.15.133:0/893783778 conn(0x7f63d8100030 msgr2=0x7f63d8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 -- 172.21.15.133:0/893783778 shutdown_connections 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.561+0000 7f63df000700 1 -- 172.21.15.133:0/893783778 wait complete. 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.562+0000 7f63df000700 1 Processor -- start 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 -- start start 2024-05-09T08:28:59.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104f20 0x7f63d81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8114880 con 0x7f63d8104560 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8114700 con 0x7f63d810a610 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63df000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63d8114a00 con 0x7f63d8104f20 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63dd000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63d7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46972/0 (socket says 172.21.15.133:46972) 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63d7e00700 1 -- 172.21.15.133:0/1606180664 learned_addr learned my addr 172.21.15.133:0/1606180664 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63dd000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:45864/0 (socket says 172.21.15.133:45864) 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.563+0000 7f63d7400700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104f20 0x7f63d81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 -- 172.21.15.133:0/1606180664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104f20 msgr2=0x7f63d81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104f20 0x7f63d81a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 -- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 msgr2=0x7f63d81a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 -- 172.21.15.133:0/1606180664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d819b160 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63d7e00700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63dd000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b770 tx=0x7f63d0007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0021020 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d0004c50 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00052b0 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.564+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d819b3f0 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.565+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d81a7710 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.565+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d8105c80 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.567+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f63d0024020 con 0x7f63d810a610 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.569+0000 7f63d4c00700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 0x7f63bc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:28:59.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.569+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f63d000f020 con 0x7f63d810a610 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.569+0000 7f63d7e00700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 0x7f63bc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.570+0000 7f63d7e00700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 0x7f63bc0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63d819bb40 tx=0x7f63c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.573+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63d009ce90 con 0x7f63d810a610 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.803+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f63d81a78e0 con 0x7f63d810a610 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.803+0000 7f63d4c00700 1 -- 172.21.15.133:0/1606180664 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f63d001de20 con 0x7f63d810a610 2024-05-09T08:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout:146028888129 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.806+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 msgr2=0x7f63bc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.806+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 0x7f63bc0a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63d819bb40 tx=0x7f63c8009260 comp rx=0 tx=0).stop 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.806+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 msgr2=0x7f63d819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.806+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b770 tx=0x7f63d0007a20 comp rx=0 tx=0).stop 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 shutdown_connections 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d810a610 0x7f63d819aa00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63d8104560 0x7f63d81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f63bc0a2d30 0x7f63bc0a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 --2- 172.21.15.133:0/1606180664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d8104f20 0x7f63d81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:28:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 >> 172.21.15.133:0/1606180664 conn(0x7f63d8100030 msgr2=0x7f63d81014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:28:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 shutdown_connections 2024-05-09T08:28:59.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:58.807+0000 7f63df000700 1 -- 172.21.15.133:0/1606180664 wait complete. 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 -- 172.21.15.133:0/3574735166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a950 msgr2=0x7f8cfc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 --2- 172.21.15.133:0/3574735166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a950 0x7f8cfc10e890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009990 tx=0x7f8ce8009ca0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 -- 172.21.15.133:0/3574735166 shutdown_connections 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 --2- 172.21.15.133:0/3574735166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc10edd0 0x7f8cfc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 --2- 172.21.15.133:0/3574735166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc06a030 0x7f8cfc06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 --2- 172.21.15.133:0/3574735166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a950 0x7f8cfc10e890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 -- 172.21.15.133:0/3574735166 >> 172.21.15.133:0/3574735166 conn(0x7f8cfc0781d0 msgr2=0x7f8cfc0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 -- 172.21.15.133:0/3574735166 shutdown_connections 2024-05-09T08:29:00.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.214+0000 7f8d03000700 1 -- 172.21.15.133:0/3574735166 wait complete. 2024-05-09T08:29:00.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.215+0000 7f8d03000700 1 Processor -- start 2024-05-09T08:29:00.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 -- start start 2024-05-09T08:29:00.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 0x7f8cfc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:00.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc10edd0 0x7f8cfc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1148a0 con 0x7f8cfc06a950 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc114720 con 0x7f8cfc10edd0 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8d03000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc114a20 con 0x7f8cfc06a030 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.216+0000 7f8cfb400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47018/0 (socket says 172.21.15.133:47018) 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 -- 172.21.15.133:0/3340812497 learned_addr learned my addr 172.21.15.133:0/3340812497 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfbe00700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 0x7f8cfc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:00.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8d01000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc10edd0 0x7f8cfc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:00.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 -- 172.21.15.133:0/3340812497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 msgr2=0x7f8cfc19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 0x7f8cfc19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 -- 172.21.15.133:0/3340812497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc10edd0 msgr2=0x7f8cfc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc10edd0 0x7f8cfc19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 -- 172.21.15.133:0/3340812497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc19fd70 con 0x7f8cfc06a950 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfbe00700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 0x7f8cfc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.217+0000 7f8cfb400700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.218+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8027070 con 0x7f8cfc06a950 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.218+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce80056d0 con 0x7f8cfc06a950 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.218+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8022e10 con 0x7f8cfc06a950 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.218+0000 7f8d03000700 1 -- 172.21.15.133:0/3340812497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc1a0000 con 0x7f8cfc06a950 2024-05-09T08:29:00.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.218+0000 7f8d03000700 1 -- 172.21.15.133:0/3340812497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc1a78e0 con 0x7f8cfc06a950 2024-05-09T08:29:00.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.219+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd4005330 con 0x7f8cfc06a950 2024-05-09T08:29:00.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.222+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8ce802a070 con 0x7f8cfc06a950 2024-05-09T08:29:00.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.223+0000 7f8cf8c00700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 0x7f8ce00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:00.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.223+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8ce80d0420 con 0x7f8cfc06a950 2024-05-09T08:29:00.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.223+0000 7f8cfbe00700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 0x7f8ce00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:00.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.224+0000 7f8cfbe00700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 0x7f8ce00a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011490 tx=0x7f8cec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:00.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.229+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8ce8013070 con 0x7f8cfc06a950 2024-05-09T08:29:00.295 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:00 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2186217755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T08:29:00.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.464+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8cd40058f0 con 0x7f8cfc06a950 2024-05-09T08:29:00.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.465+0000 7f8cf8c00700 1 -- 172.21.15.133:0/3340812497 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ce809d280 con 0x7f8cfc06a950 2024-05-09T08:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:34359738456 2024-05-09T08:29:00.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 msgr2=0x7f8ce00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:00.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 0x7f8ce00a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8cec011490 tx=0x7f8cec009260 comp rx=0 tx=0).stop 2024-05-09T08:29:00.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 msgr2=0x7f8cfc19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:00.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8000c00 tx=0x7f8ce800bab0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 shutdown_connections 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8cfc10edd0 0x7f8cfc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8ce00a2d30 0x7f8ce00a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cfc06a030 0x7f8cfc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 --2- 172.21.15.133:0/3340812497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8cfc06a950 0x7f8cfc19b2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.470+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 >> 172.21.15.133:0/3340812497 conn(0x7f8cfc0781d0 msgr2=0x7f8cfc110220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.471+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 shutdown_connections 2024-05-09T08:29:00.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.471+0000 7f8cf2000700 1 -- 172.21.15.133:0/3340812497 wait complete. 2024-05-09T08:29:00.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:00 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2186217755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T08:29:00.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:00 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2186217755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T08:29:01.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:01 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3340812497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T08:29:01.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:01 smithi133 ceph-mon[29610]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:01.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:01 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3340812497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T08:29:01.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:01 smithi160 ceph-mon[34070]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:01.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:01 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3340812497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T08:29:01.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:01 smithi188 ceph-mon[34645]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:01.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 -- 172.21.15.133:0/3329257784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074e70 msgr2=0x7f3f68112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/3329257784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074e70 0x7f3f68112fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098f0 tx=0x7f3f54009c00 comp rx=0 tx=0).stop 2024-05-09T08:29:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 -- 172.21.15.133:0/3329257784 shutdown_connections 2024-05-09T08:29:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/3329257784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f681134f0 0x7f3f681158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/3329257784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f68074550 0x7f3f68074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/3329257784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074e70 0x7f3f68112fb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 -- 172.21.15.133:0/3329257784 >> 172.21.15.133:0/3329257784 conn(0x7f3f68079480 msgr2=0x7f3f68079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 -- 172.21.15.133:0/3329257784 shutdown_connections 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.916+0000 7f3f6ec00700 1 -- 172.21.15.133:0/3329257784 wait complete. 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.917+0000 7f3f6ec00700 1 Processor -- start 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.917+0000 7f3f6ec00700 1 -- start start 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f68074e70 0x7f3f6819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 0x7f3f681a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68118c30 con 0x7f3f68074550 2024-05-09T08:29:01.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68118ab0 con 0x7f3f681134f0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6ec00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68118db0 con 0x7f3f68074e70 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f67e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f67e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46988/0 (socket says 172.21.15.133:46988) 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f67e00700 1 -- 172.21.15.133:0/2186217755 learned_addr learned my addr 172.21.15.133:0/2186217755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f67400700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f68074e70 0x7f3f6819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.918+0000 7f3f6cc00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 0x7f3f681a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 -- 172.21.15.133:0/2186217755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f68074e70 msgr2=0x7f3f6819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f68074e70 0x7f3f6819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 -- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 msgr2=0x7f3f6819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 -- 172.21.15.133:0/2186217755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681a4170 con 0x7f3f681134f0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f67e00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6cc00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 0x7f3f681a3a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ef90 tx=0x7f3f6000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:01.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600092e0 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a4400 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f60004bb0 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60004280 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.919+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681abce0 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.920+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68069010 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.922+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3f6001b030 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.924+0000 7f3f64c00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 0x7f3f4c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.924+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3f600cf6e0 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.924+0000 7f3f67e00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 0x7f3f4c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.925+0000 7f3f67e00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 0x7f3f4c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005950 tx=0x7f3f58009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:28:59.926+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f60022020 con 0x7f3f681134f0 2024-05-09T08:29:01.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.156+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f3f681abeb0 con 0x7f3f681134f0 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stdout:257698037805 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.157+0000 7f3f64c00700 1 -- 172.21.15.133:0/2186217755 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3f60010040 con 0x7f3f681134f0 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 msgr2=0x7f3f4c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 0x7f3f4c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005950 tx=0x7f3f58009310 comp rx=0 tx=0).stop 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 msgr2=0x7f3f681a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:01.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 0x7f3f681a3a30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ef90 tx=0x7f3f6000c3a0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 shutdown_connections 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f681134f0 0x7f3f681a3a30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68074550 0x7f3f6819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3f4c0a2d30 0x7f3f4c0a51f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.160+0000 7f3f6ec00700 1 --2- 172.21.15.133:0/2186217755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f68074e70 0x7f3f6819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.161+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 >> 172.21.15.133:0/2186217755 conn(0x7f3f68079480 msgr2=0x7f3f68112320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.161+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 shutdown_connections 2024-05-09T08:29:01.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:00.161+0000 7f3f6ec00700 1 -- 172.21.15.133:0/2186217755 wait complete. 2024-05-09T08:29:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:02.013 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:03.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037803 got 257698037805 for osd.8 2024-05-09T08:29:03.086 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:03.423 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:03.843 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:03 smithi133 ceph-mon[29610]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 -- 172.21.15.133:0/4245213508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 msgr2=0x7fc018115820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:03.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 --2- 172.21.15.133:0/4245213508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc018115820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc01000b0a0 tx=0x7fc01000b3b0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 -- 172.21.15.133:0/4245213508 shutdown_connections 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 --2- 172.21.15.133:0/4245213508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc018115820 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 --2- 172.21.15.133:0/4245213508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01810e950 0x7fc01810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 --2- 172.21.15.133:0/4245213508 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc018108950 0x7fc018108dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 -- 172.21.15.133:0/4245213508 >> 172.21.15.133:0/4245213508 conn(0x7fc018075af0 msgr2=0x7fc018107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 -- 172.21.15.133:0/4245213508 shutdown_connections 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.925+0000 7fc020600700 1 -- 172.21.15.133:0/4245213508 wait complete. 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.926+0000 7fc020600700 1 Processor -- start 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.926+0000 7fc020600700 1 -- start start 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc018108950 0x7fc01819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc01810e950 0x7fc0181a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:03.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc018118d30 con 0x7fc018109310 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc018118bb0 con 0x7fc01810e950 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc020600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc018118eb0 con 0x7fc018108950 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc01d200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc01d200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46486/0 (socket says 172.21.15.133:46486) 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc01d200700 1 -- 172.21.15.133:0/927247805 learned_addr learned my addr 172.21.15.133:0/927247805 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc01dc00700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc018108950 0x7fc01819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.927+0000 7fc01e600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc01810e950 0x7fc0181a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 -- 172.21.15.133:0/927247805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc018108950 msgr2=0x7fc01819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc018108950 0x7fc01819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 -- 172.21.15.133:0/927247805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc01810e950 msgr2=0x7fc0181a3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc01810e950 0x7fc0181a3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:03.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 -- 172.21.15.133:0/927247805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0181a40b0 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc01d200700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e9a0 tx=0x7fc00400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00400ca10 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00400cbb0 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004018680 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.928+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0181a43a0 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.929+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0181abc20 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.930+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc018069010 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.932+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc004010430 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.934+0000 7fc016000700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 0x7fc0000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.934+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc0040cf380 con 0x7fc018109310 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.934+0000 7fc01dc00700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 0x7fc0000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:03.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.935+0000 7fc01dc00700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 0x7fc0000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009be0 tx=0x7fc00c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:03.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:03.939+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc00409c150 con 0x7fc018109310 2024-05-09T08:29:04.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:03 smithi160 ceph-mon[34070]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:04.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.175+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc0181abe20 con 0x7fc018109310 2024-05-09T08:29:04.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.176+0000 7fc016000700 1 -- 172.21.15.133:0/927247805 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc00409ba50 con 0x7fc018109310 2024-05-09T08:29:04.176 INFO:teuthology.orchestra.run.smithi133.stdout:111669149768 2024-05-09T08:29:04.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.178+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 msgr2=0x7fc0000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:04.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.178+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 0x7fc0000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009be0 tx=0x7fc00c009260 comp rx=0 tx=0).stop 2024-05-09T08:29:04.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.178+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 msgr2=0x7fc01819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:04.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.178+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e9a0 tx=0x7fc00400ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 shutdown_connections 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc01810e950 0x7fc0181a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc0000a2d30 0x7fc0000a51f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc018108950 0x7fc01819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 --2- 172.21.15.133:0/927247805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc018109310 0x7fc01819f5e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 >> 172.21.15.133:0/927247805 conn(0x7fc018075af0 msgr2=0x7fc018075fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 shutdown_connections 2024-05-09T08:29:04.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:04.179+0000 7fc020600700 1 -- 172.21.15.133:0/927247805 wait complete. 2024-05-09T08:29:04.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:03 smithi188 ceph-mon[34645]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:05.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:04 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/927247805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T08:29:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:04 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/927247805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T08:29:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:04 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/927247805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T08:29:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:05 smithi133 ceph-mon[29610]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:05.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:05 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3355116256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T08:29:06.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:05 smithi160 ceph-mon[34070]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:06.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:05 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3355116256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T08:29:06.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:05 smithi188 ceph-mon[34645]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-05-09T08:29:06.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:05 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3355116256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T08:29:06.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888123 got 146028888129 for osd.4 2024-05-09T08:29:06.227 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:06.322 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738456 for osd.0 2024-05-09T08:29:06.322 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:06.891 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:06 smithi133 ceph-mon[29610]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-09T08:29:07.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 -- 172.21.15.133:0/4070994368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 msgr2=0x7f5e3810a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 --2- 172.21.15.133:0/4070994368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3810a8a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5e280098f0 tx=0x7f5e28009c00 comp rx=0 tx=0).stop 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 -- 172.21.15.133:0/4070994368 shutdown_connections 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 --2- 172.21.15.133:0/4070994368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 0x7f5e38111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 --2- 172.21.15.133:0/4070994368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3810a8a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 --2- 172.21.15.133:0/4070994368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 -- 172.21.15.133:0/4070994368 >> 172.21.15.133:0/4070994368 conn(0x7f5e380fff00 msgr2=0x7f5e38102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 -- 172.21.15.133:0/4070994368 shutdown_connections 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.921+0000 7f5e37e00700 1 -- 172.21.15.133:0/4070994368 wait complete. 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 Processor -- start 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 -- start start 2024-05-09T08:29:07.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 0x7f5e381a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38114790 con 0x7f5e38104df0 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38114610 con 0x7f5e3810a4c0 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.923+0000 7f5e37e00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38114910 con 0x7f5e38104430 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 0x7f5e381a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:46588/0 (socket says 172.21.15.133:46588) 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 -- 172.21.15.133:0/3763950075 learned_addr learned my addr 172.21.15.133:0/3763950075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e37400700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 -- 172.21.15.133:0/3763950075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 msgr2=0x7f5e3819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 -- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 msgr2=0x7f5e381a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 0x7f5e381a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 -- 172.21.15.133:0/3763950075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3819b1b0 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.924+0000 7f5e36a00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005950 tx=0x7f5e2800fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.925+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005140 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.925+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e2800b560 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.925+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3819b440 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.925+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28003ae0 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.925+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381a7760 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.926+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e38105b70 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.928+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5e2800ba80 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.929+0000 7f5e3ca00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 0x7f5e1c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.930+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5e280d0150 con 0x7f5e38104430 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.930+0000 7f5e36000700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 0x7f5e1c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.930+0000 7f5e36000700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 0x7f5e1c0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5e38102540 tx=0x7f5e24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:07.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:06.934+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5e28013030 con 0x7f5e38104430 2024-05-09T08:29:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.160+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5e38069010 con 0x7f5e38104430 2024-05-09T08:29:07.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.160+0000 7f5e3ca00700 1 -- 172.21.15.133:0/3763950075 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5e2809cfb0 con 0x7f5e38104430 2024-05-09T08:29:07.160 INFO:teuthology.orchestra.run.smithi133.stdout:347892351005 2024-05-09T08:29:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:06 smithi160 ceph-mon[34070]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 msgr2=0x7f5e1c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 0x7f5e1c0a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5e38102540 tx=0x7f5e24009260 comp rx=0 tx=0).stop 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 msgr2=0x7f5e381a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5e28005950 tx=0x7f5e2800fb20 comp rx=0 tx=0).stop 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 shutdown_connections 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e3810a4c0 0x7f5e3819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5e1c0a2cb0 0x7f5e1c0a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e38104430 0x7f5e381a0890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 --2- 172.21.15.133:0/3763950075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e38104df0 0x7f5e381a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 >> 172.21.15.133:0/3763950075 conn(0x7f5e380fff00 msgr2=0x7f5e38100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 shutdown_connections 2024-05-09T08:29:07.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:07.164+0000 7f5e37e00700 1 -- 172.21.15.133:0/3763950075 wait complete. 2024-05-09T08:29:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:06 smithi188 ceph-mon[34645]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2024-05-09T08:29:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 -- 172.21.15.133:0/334291010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103d30 msgr2=0x7f190c10c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 --2- 172.21.15.133:0/334291010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103d30 0x7f190c10c220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18f80098f0 tx=0x7f18f8009c00 comp rx=0 tx=0).stop 2024-05-09T08:29:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 -- 172.21.15.133:0/334291010 shutdown_connections 2024-05-09T08:29:07.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 --2- 172.21.15.133:0/334291010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 0x7f190c10ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 --2- 172.21.15.133:0/334291010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103410 0x7f190c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 --2- 172.21.15.133:0/334291010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103d30 0x7f190c10c220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 -- 172.21.15.133:0/334291010 >> 172.21.15.133:0/334291010 conn(0x7f190c0fcc40 msgr2=0x7f190c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 -- 172.21.15.133:0/334291010 shutdown_connections 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.022+0000 7f1912400700 1 -- 172.21.15.133:0/334291010 wait complete. 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.023+0000 7f1912400700 1 Processor -- start 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.023+0000 7f1912400700 1 -- start start 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103d30 0x7f190c1015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 0x7f190c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f190c112560 con 0x7f190c10c840 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1123e0 con 0x7f190c103410 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1912400700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1126e0 con 0x7f190c103d30 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:55420/0 (socket says 172.21.15.133:55420) 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 -- 172.21.15.133:0/3355116256 learned_addr learned my addr 172.21.15.133:0/3355116256 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:07.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f190be00700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103d30 0x7f190c1015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911a00700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 0x7f190c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 -- 172.21.15.133:0/3355116256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103d30 msgr2=0x7f190c1015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103d30 0x7f190c1015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 -- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 msgr2=0x7f190c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 0x7f190c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.024+0000 7f1911000700 1 -- 172.21.15.133:0/3355116256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1a6f90 con 0x7f190c103410 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1911000700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9a0 tx=0x7f190000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:07.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000cbd0 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f190000cd70 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900018750 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a7280 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.025+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a7740 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.027+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c06a8e0 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.029+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1900010430 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.030+0000 7f1909600700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 0x7f18f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.030+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f19000cf4e0 con 0x7f190c103410 2024-05-09T08:29:07.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.030+0000 7f190be00700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 0x7f18f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.031+0000 7f190be00700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 0x7f18f40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005e90 tx=0x7f18f8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.034+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f190009c2b0 con 0x7f190c103410 2024-05-09T08:29:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.267+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f190c102550 con 0x7f190c103410 2024-05-09T08:29:07.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.268+0000 7f1909600700 1 -- 172.21.15.133:0/3355116256 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f190009bbb0 con 0x7f190c103410 2024-05-09T08:29:07.720 INFO:teuthology.orchestra.run.smithi133.stdout:167503724607 2024-05-09T08:29:07.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 msgr2=0x7f18f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 0x7f18f40a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f18f8005e90 tx=0x7f18f8019040 comp rx=0 tx=0).stop 2024-05-09T08:29:07.721 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 msgr2=0x7f190c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9a0 tx=0x7f190000ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 shutdown_connections 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.271+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190c10c840 0x7f190c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f190c103410 0x7f190c101070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f18f40a2cb0 0x7f18f40a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 --2- 172.21.15.133:0/3355116256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f190c103d30 0x7f190c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 >> 172.21.15.133:0/3355116256 conn(0x7f190c0fcc40 msgr2=0x7f190c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:07.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 shutdown_connections 2024-05-09T08:29:07.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:05.272+0000 7f1912400700 1 -- 172.21.15.133:0/3355116256 wait complete. 2024-05-09T08:29:07.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:07 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3763950075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T08:29:07.949 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:08.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:07 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3763950075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T08:29:08.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:07 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3763950075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T08:29:08.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149763 got 111669149768 for osd.3 2024-05-09T08:29:08.565 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:09.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:08 smithi133 ceph-mon[29610]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-09T08:29:09.084 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:08 smithi188 ceph-mon[34645]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-09T08:29:09.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:08 smithi160 ceph-mon[34070]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-05-09T08:29:09.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:09 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3198350699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T08:29:10.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:09 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3198350699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T08:29:10.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:09 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3198350699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T08:29:10.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 -- 172.21.15.133:0/3684411045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60077170 msgr2=0x7fcc60112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:10.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 --2- 172.21.15.133:0/3684411045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60077170 0x7fcc60112c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009990 tx=0x7fcc4c009ca0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 -- 172.21.15.133:0/3684411045 shutdown_connections 2024-05-09T08:29:10.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 --2- 172.21.15.133:0/3684411045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 --2- 172.21.15.133:0/3684411045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60076850 0x7fcc60076c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 --2- 172.21.15.133:0/3684411045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60077170 0x7fcc60112c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 -- 172.21.15.133:0/3684411045 >> 172.21.15.133:0/3684411045 conn(0x7fcc60100060 msgr2=0x7fcc60102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 -- 172.21.15.133:0/3684411045 shutdown_connections 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.350+0000 7fcc68200700 1 -- 172.21.15.133:0/3684411045 wait complete. 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.351+0000 7fcc68200700 1 Processor -- start 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 -- start start 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60077170 0x7fcc6019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60118cc0 con 0x7fcc60077170 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60118b40 con 0x7fcc601131d0 2024-05-09T08:29:10.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc68200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60118e40 con 0x7fcc60076850 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc65800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc65800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:46608/0 (socket says 172.21.15.133:46608) 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc65800700 1 -- 172.21.15.133:0/3198350699 learned_addr learned my addr 172.21.15.133:0/3198350699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.352+0000 7fcc64e00700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60077170 0x7fcc6019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc66200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 -- 172.21.15.133:0/3198350699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 msgr2=0x7fcc601a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 -- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60077170 msgr2=0x7fcc6019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60077170 0x7fcc6019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 -- 172.21.15.133:0/3198350699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc601a4120 con 0x7fcc60076850 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc66200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc65800700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e9a0 tx=0x7fcc5400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400cbd0 con 0x7fcc60076850 2024-05-09T08:29:10.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5400cd70 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc601a4410 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54018750 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.353+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc601abc90 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.355+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc60069010 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.356+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcc54010430 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.358+0000 7fcc5e000700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 0x7fcc480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.358+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcc540cf460 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.358+0000 7fcc64e00700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 0x7fcc480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.359+0000 7fcc64e00700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 0x7fcc480a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b2e0 tx=0x7fcc4c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.362+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc5409c290 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.589+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcc6002db80 con 0x7fcc60076850 2024-05-09T08:29:10.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.589+0000 7fcc5e000700 1 -- 172.21.15.133:0/3198350699 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc5409bb90 con 0x7fcc60076850 2024-05-09T08:29:10.517 INFO:teuthology.orchestra.run.smithi133.stdout:292057776167 2024-05-09T08:29:10.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 msgr2=0x7fcc480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:10.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 0x7fcc480a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b2e0 tx=0x7fcc4c005960 comp rx=0 tx=0).stop 2024-05-09T08:29:10.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 msgr2=0x7fcc6019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e9a0 tx=0x7fcc5400ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 shutdown_connections 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc601131d0 0x7fcc601a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fcc480a2cb0 0x7fcc480a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc60076850 0x7fcc6019f110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 --2- 172.21.15.133:0/3198350699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc60077170 0x7fcc6019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 >> 172.21.15.133:0/3198350699 conn(0x7fcc60100060 msgr2=0x7fcc60102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 shutdown_connections 2024-05-09T08:29:10.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:09.593+0000 7fcc68200700 1 -- 172.21.15.133:0/3198350699 wait complete. 2024-05-09T08:29:10.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351000 got 347892351005 for osd.11 2024-05-09T08:29:10.896 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:10 smithi160 ceph-mon[34070]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T08:29:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:10 smithi133 ceph-mon[29610]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T08:29:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:10 smithi188 ceph-mon[34645]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2024-05-09T08:29:12.283 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724602 got 167503724607 for osd.5 2024-05-09T08:29:12.284 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:13.792 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:13 smithi133 ceph-mon[29610]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:13 smithi160 ceph-mon[34070]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:14.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:13 smithi188 ceph-mon[34645]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.975+0000 7f616d200700 1 -- 172.21.15.133:0/1839904935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074e70 msgr2=0x7f6168112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.975+0000 7f616d200700 1 --2- 172.21.15.133:0/1839904935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074e70 0x7f6168112fb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6150009990 tx=0x7f6150009ca0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 -- 172.21.15.133:0/1839904935 shutdown_connections 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 --2- 172.21.15.133:0/1839904935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61681134f0 0x7f61681158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 --2- 172.21.15.133:0/1839904935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6168074550 0x7f6168074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 --2- 172.21.15.133:0/1839904935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074e70 0x7f6168112fb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 -- 172.21.15.133:0/1839904935 >> 172.21.15.133:0/1839904935 conn(0x7f61680794d0 msgr2=0x7f61680798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 -- 172.21.15.133:0/1839904935 shutdown_connections 2024-05-09T08:29:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.976+0000 7f616d200700 1 -- 172.21.15.133:0/1839904935 wait complete. 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 Processor -- start 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 -- start start 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 0x7f616819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61681134f0 0x7f61681a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6168118cd0 con 0x7f6168074550 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6168118b50 con 0x7f6168074e70 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f616d200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6168118e50 con 0x7f61681134f0 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.977+0000 7f6165e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 0x7f616819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:14.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f6165e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45642/0 (socket says 172.21.15.133:45642) 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f6165e00700 1 -- 172.21.15.133:0/3570950584 learned_addr learned my addr 172.21.15.133:0/3570950584 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f6166800700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61681134f0 0x7f61681a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 -- 172.21.15.133:0/3570950584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61681134f0 msgr2=0x7f61681a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61681134f0 0x7f61681a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 -- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 msgr2=0x7f616819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 -- 172.21.15.133:0/3570950584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681a4130 con 0x7f6168074e70 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f6165e00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:14.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.978+0000 7f615fe00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 0x7f616819f660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6150000c00 tx=0x7f615000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.979+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150026070 con 0x7f6168074e70 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.979+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6150003d70 con 0x7f6168074e70 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.979+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681a43c0 con 0x7f6168074e70 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.979+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000fdc0 con 0x7f6168074e70 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.979+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681abca0 con 0x7f6168074e70 2024-05-09T08:29:14.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.980+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6168069010 con 0x7f6168074e70 2024-05-09T08:29:14.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.982+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6150029070 con 0x7f6168074e70 2024-05-09T08:29:14.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.983+0000 7f615ea00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 0x7f614c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.983+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f61500d0300 con 0x7f6168074e70 2024-05-09T08:29:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.983+0000 7f6165e00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 0x7f614c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:14.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.984+0000 7f6165e00700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 0x7f614c0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6158011590 tx=0x7f6158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:14.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:14.985+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6150030020 con 0x7f6168074e70 2024-05-09T08:29:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.222+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f61681abe70 con 0x7f6168074e70 2024-05-09T08:29:15.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.222+0000 7f615ea00700 1 -- 172.21.15.133:0/3570950584 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6150013070 con 0x7f6168074e70 2024-05-09T08:29:15.223 INFO:teuthology.orchestra.run.smithi133.stdout:77309411409 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 msgr2=0x7f614c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 0x7f614c0a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6158011590 tx=0x7f6158009260 comp rx=0 tx=0).stop 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 msgr2=0x7f616819f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 0x7f616819f660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6150000c00 tx=0x7f615000bab0 comp rx=0 tx=0).stop 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 shutdown_connections 2024-05-09T08:29:15.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f614c0a2cb0 0x7f614c0a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61681134f0 0x7f61681a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074550 0x7f616819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 --2- 172.21.15.133:0/3570950584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6168074e70 0x7f616819f660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.225+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 >> 172.21.15.133:0/3570950584 conn(0x7f61680794d0 msgr2=0x7f6168112240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.226+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 shutdown_connections 2024-05-09T08:29:15.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:15.226+0000 7f616d200700 1 -- 172.21.15.133:0/3570950584 wait complete. 2024-05-09T08:29:15.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:15 smithi133 ceph-mon[29610]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:15.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:15 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3570950584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T08:29:16.123 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776167 for osd.9 2024-05-09T08:29:16.123 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:16.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:15 smithi160 ceph-mon[34070]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:16.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:15 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3570950584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T08:29:16.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:15 smithi188 ceph-mon[34645]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:16.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:15 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3570950584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T08:29:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.338+0000 7fd6b0600700 1 -- 172.21.15.133:0/2621885302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a8107660 msgr2=0x7fd6a810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.338+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2621885302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a8107660 0x7fd6a810f230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b0a0 tx=0x7fd6a000b3b0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 -- 172.21.15.133:0/2621885302 shutdown_connections 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2621885302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a8107660 0x7fd6a810f230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2621885302 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a81025d0 0x7fd6a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2621885302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 -- 172.21.15.133:0/2621885302 >> 172.21.15.133:0/2621885302 conn(0x7fd6a8079480 msgr2=0x7fd6a8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 -- 172.21.15.133:0/2621885302 shutdown_connections 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.339+0000 7fd6b0600700 1 -- 172.21.15.133:0/2621885302 wait complete. 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 Processor -- start 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 -- start start 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a81a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 0x7fd6a819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112690 con 0x7fd6a81025d0 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112510 con 0x7fd6a8107660 2024-05-09T08:29:16.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.340+0000 7fd6b0600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112810 con 0x7fd6a8102f80 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6adc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ad200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a81a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6adc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45658/0 (socket says 172.21.15.133:45658) 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6adc00700 1 -- 172.21.15.133:0/2334088687 learned_addr learned my addr 172.21.15.133:0/2334088687 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 0x7fd6a819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 -- 172.21.15.133:0/2334088687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 msgr2=0x7fd6a81a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a81a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 -- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 msgr2=0x7fd6a81a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ae600700 1 -- 172.21.15.133:0/2334088687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a819b400 con 0x7fd6a8107660 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6ad200700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a81a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:16.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.341+0000 7fd6adc00700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6ae600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 0x7fd6a819ab20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b070 tx=0x7fd6a0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0021070 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0007b80 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a819b690 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a81a79d0 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.342+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0005040 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.343+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a8103a60 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.346+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd6a0024070 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.347+0000 7fd6a6000700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 0x7fd6900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.347+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd6a000f020 con 0x7fd6a8107660 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.347+0000 7fd6adc00700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 0x7fd6900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.348+0000 7fd6adc00700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 0x7fd6900a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd69c011630 tx=0x7fd69c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.349+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6a002b020 con 0x7fd6a8107660 2024-05-09T08:29:16.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.581+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd6a81a7ba0 con 0x7fd6a8107660 2024-05-09T08:29:16.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.582+0000 7fd6a6000700 1 -- 172.21.15.133:0/2334088687 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6a0007d30 con 0x7fd6a8107660 2024-05-09T08:29:16.582 INFO:teuthology.orchestra.run.smithi133.stdout:313532612642 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 msgr2=0x7fd6900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 0x7fd6900a51f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd69c011630 tx=0x7fd69c009260 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 msgr2=0x7fd6a819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 0x7fd6a819ab20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b070 tx=0x7fd6a0004c80 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 shutdown_connections 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6a8107660 0x7fd6a819ab20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6a81025d0 0x7fd6a81a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd6900a2d30 0x7fd6900a51f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 --2- 172.21.15.133:0/2334088687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6a8102f80 0x7fd6a81a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.585+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 >> 172.21.15.133:0/2334088687 conn(0x7fd6a8079480 msgr2=0x7fd6a81001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.586+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 shutdown_connections 2024-05-09T08:29:16.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.586+0000 7fd6b0600700 1 -- 172.21.15.133:0/2334088687 wait complete. 2024-05-09T08:29:16.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 -- 172.21.15.133:0/2184523047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a030 msgr2=0x7fb3b006a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 --2- 172.21.15.133:0/2184523047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a030 0x7fb3b006a490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0098f0 tx=0x7fb39c009c00 comp rx=0 tx=0).stop 2024-05-09T08:29:16.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 -- 172.21.15.133:0/2184523047 shutdown_connections 2024-05-09T08:29:16.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 --2- 172.21.15.133:0/2184523047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b006a9d0 0x7fb3b010f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 --2- 172.21.15.133:0/2184523047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b01083a0 0x7fb3b0108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 --2- 172.21.15.133:0/2184523047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a030 0x7fb3b006a490 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 -- 172.21.15.133:0/2184523047 >> 172.21.15.133:0/2184523047 conn(0x7fb3b00781d0 msgr2=0x7fb3b00785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 -- 172.21.15.133:0/2184523047 shutdown_connections 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.444+0000 7fb3b8200700 1 -- 172.21.15.133:0/2184523047 wait complete. 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.445+0000 7fb3b8200700 1 Processor -- start 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 -- start start 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a9d0 0x7fb3b0101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b01083a0 0x7fb3b0101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0112650 con 0x7fb3b01083a0 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01124d0 con 0x7fb3b006a030 2024-05-09T08:29:16.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b8200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01127d0 con 0x7fb3b006a9d0 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b5800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b6200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b01083a0 0x7fb3b0101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b5800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39472/0 (socket says 172.21.15.133:39472) 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.446+0000 7fb3b5800700 1 -- 172.21.15.133:0/4193959737 learned_addr learned my addr 172.21.15.133:0/4193959737 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b4e00700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a9d0 0x7fb3b0101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 -- 172.21.15.133:0/4193959737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a9d0 msgr2=0x7fb3b0101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a9d0 0x7fb3b0101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 -- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b01083a0 msgr2=0x7fb3b0101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b01083a0 0x7fb3b0101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 -- 172.21.15.133:0/4193959737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b01a70d0 con 0x7fb3b006a030 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3b5800700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c110 tx=0x7fb3a400c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.447+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40092e0 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.448+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b01a7360 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.448+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a4004bb0 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.448+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4007dc0 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.448+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01a7870 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.449+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b0069010 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.451+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb3a401a020 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.452+0000 7fb3ae000700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 0x7fb3980a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.452+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb3a40cf020 con 0x7fb3b006a030 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.452+0000 7fb3b4e00700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 0x7fb3980a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.453+0000 7fb3b4e00700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 0x7fb3980a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009810 tx=0x7fb39c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.455+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3a4010040 con 0x7fb3b006a030 2024-05-09T08:29:16.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.689+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb3b002db80 con 0x7fb3b006a030 2024-05-09T08:29:16.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.690+0000 7fb3ae000700 1 -- 172.21.15.133:0/4193959737 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb3a409be50 con 0x7fb3b006a030 2024-05-09T08:29:16.689 INFO:teuthology.orchestra.run.smithi133.stdout:201863462971 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 msgr2=0x7fb3980a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 0x7fb3980a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009810 tx=0x7fb39c019040 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 msgr2=0x7fb3b01012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c110 tx=0x7fb3a400c420 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 shutdown_connections 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3b01083a0 0x7fb3b0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3b006a030 0x7fb3b01012f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb3980a2cb0 0x7fb3980a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 --2- 172.21.15.133:0/4193959737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b006a9d0 0x7fb3b0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 >> 172.21.15.133:0/4193959737 conn(0x7fb3b00781d0 msgr2=0x7fb3b00fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 shutdown_connections 2024-05-09T08:29:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7fb3b8200700 1 -- 172.21.15.133:0/4193959737 wait complete. 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 -- 172.21.15.133:0/514061770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 msgr2=0x7f826010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 --2- 172.21.15.133:0/514061770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826010f160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f825800b0d0 tx=0x7f825800b3e0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 -- 172.21.15.133:0/514061770 shutdown_connections 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 --2- 172.21.15.133:0/514061770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826010f160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 --2- 172.21.15.133:0/514061770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82601024b0 0x7f8260102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 --2- 172.21.15.133:0/514061770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8260102e60 0x7f8260106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 -- 172.21.15.133:0/514061770 >> 172.21.15.133:0/514061770 conn(0x7f82600fdda0 msgr2=0x7f82601001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 -- 172.21.15.133:0/514061770 shutdown_connections 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.684+0000 7f8265200700 1 -- 172.21.15.133:0/514061770 wait complete. 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.685+0000 7f8265200700 1 Processor -- start 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 -- start start 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8260102e60 0x7f82601a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82601125d0 con 0x7f8260102e60 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112450 con 0x7f82601024b0 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f8265200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112750 con 0x7f8260107590 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f825f400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f825ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8260102e60 0x7f82601a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f825f400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39500/0 (socket says 172.21.15.133:39500) 2024-05-09T08:29:16.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f825f400700 1 -- 172.21.15.133:0/3898471247 learned_addr learned my addr 172.21.15.133:0/3898471247 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.686+0000 7f825fe00700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 -- 172.21.15.133:0/3898471247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 msgr2=0x7f826019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 -- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8260102e60 msgr2=0x7f82601a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8260102e60 0x7f82601a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 -- 172.21.15.133:0/3898471247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826019b1d0 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f825f400700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9a0 tx=0x7f825000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825000ca10 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826019b4c0 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.687+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82601a7730 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.688+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f825000cbb0 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.688+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250018680 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.689+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240005330 con 0x7f82601024b0 2024-05-09T08:29:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.691+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8250018820 con 0x7f82601024b0 2024-05-09T08:29:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7f8257e00700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 0x7f82440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.692+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f82500cf2b0 con 0x7f82601024b0 2024-05-09T08:29:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7f825ea00700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 0x7f82440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.693+0000 7f825ea00700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 0x7f82440a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8260101660 tx=0x7f824c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:16.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.696+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f825009c080 con 0x7f82601024b0 2024-05-09T08:29:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:16 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2334088687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T08:29:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:16 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4193959737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T08:29:16.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:16 smithi133 ceph-mon[29610]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:16.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.928+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f82400058f0 con 0x7f82601024b0 2024-05-09T08:29:16.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.929+0000 7f8257e00700 1 -- 172.21.15.133:0/3898471247 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f825009b980 con 0x7f82601024b0 2024-05-09T08:29:16.929 INFO:teuthology.orchestra.run.smithi133.stdout:223338299445 2024-05-09T08:29:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 msgr2=0x7f82440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 0x7f82440a51f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8260101660 tx=0x7f824c009260 comp rx=0 tx=0).stop 2024-05-09T08:29:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 msgr2=0x7f82601a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f825000e9a0 tx=0x7f825000ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 shutdown_connections 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f82440a2d30 0x7f82440a51f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260107590 0x7f826019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82601024b0 0x7f82601a0860 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 --2- 172.21.15.133:0/3898471247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8260102e60 0x7f82601a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.932+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 >> 172.21.15.133:0/3898471247 conn(0x7f82600fdda0 msgr2=0x7f826010d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.933+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 shutdown_connections 2024-05-09T08:29:16.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:16.933+0000 7f8265200700 1 -- 172.21.15.133:0/3898471247 wait complete. 2024-05-09T08:29:17.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:16 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2334088687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T08:29:17.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:16 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4193959737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T08:29:17.064 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:16 smithi160 ceph-mon[34070]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:16 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2334088687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T08:29:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:16 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4193959737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T08:29:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:16 smithi188 ceph-mon[34645]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:17.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:17 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3898471247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T08:29:18.070 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299445 for osd.7 2024-05-09T08:29:18.070 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:18.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:17 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3898471247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T08:29:18.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:17 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3898471247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T08:29:18.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612636 got 313532612642 for osd.10 2024-05-09T08:29:18.873 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:19.103 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:18 smithi133 ceph-mon[29610]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:18 smithi160 ceph-mon[34070]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:19.171 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411409 for osd.2 2024-05-09T08:29:19.171 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:18 smithi188 ceph-mon[34645]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:19.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462964 got 201863462971 for osd.6 2024-05-09T08:29:19.839 DEBUG:teuthology.parallel:result is None 2024-05-09T08:29:19.839 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T08:29:19.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T08:29:20.259 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:22.061 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:21 smithi133 ceph-mon[29610]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:22.062 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.062 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:21 smithi160 ceph-mon[34070]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:21 smithi188 ceph-mon[34645]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:22.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 -- 172.21.15.133:0/3626711753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806b1e0 msgr2=0x7fea3810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 --2- 172.21.15.133:0/3626711753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806b1e0 0x7fea3810e880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea24009990 tx=0x7fea24009ca0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 -- 172.21.15.133:0/3626711753 shutdown_connections 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 --2- 172.21.15.133:0/3626711753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 0x7fea381111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 --2- 172.21.15.133:0/3626711753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806a8c0 0x7fea3806aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.597+0000 7fea3f800700 1 --2- 172.21.15.133:0/3626711753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806b1e0 0x7fea3810e880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.598+0000 7fea3f800700 1 -- 172.21.15.133:0/3626711753 >> 172.21.15.133:0/3626711753 conn(0x7fea38078190 msgr2=0x7fea380785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.598+0000 7fea3f800700 1 -- 172.21.15.133:0/3626711753 shutdown_connections 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.598+0000 7fea3f800700 1 -- 172.21.15.133:0/3626711753 wait complete. 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 Processor -- start 2024-05-09T08:29:22.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 -- start start 2024-05-09T08:29:22.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 0x7fea381a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:22.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 0x7fea381a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:22.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 0x7fea3819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea38114840 con 0x7fea3806a8c0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fea381146c0 con 0x7fea3810edc0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.599+0000 7fea3f800700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea381149c0 con 0x7fea3806b1e0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea3ce00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 0x7fea381a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea3ce00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 0x7fea381a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45730/0 (socket says 172.21.15.133:45730) 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea3ce00700 1 -- 172.21.15.133:0/2876451261 learned_addr learned my addr 172.21.15.133:0/2876451261 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 0x7fea381a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea3d800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 0x7fea3819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 -- 172.21.15.133:0/2876451261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 msgr2=0x7fea3819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 0x7fea3819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 -- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 msgr2=0x7fea381a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:22.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 0x7fea381a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.600+0000 7fea2fe00700 1 -- 172.21.15.133:0/2876451261 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3819b400 con 0x7fea3806b1e0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea2fe00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 0x7fea381a0f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea240059d0 tx=0x7fea2400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24027070 con 0x7fea3806b1e0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3819b690 con 0x7fea3806b1e0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea24003d70 con 0x7fea3806b1e0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400f460 con 0x7fea3806b1e0 2024-05-09T08:29:22.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.601+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a79d0 con 0x7fea3806b1e0 2024-05-09T08:29:22.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.602+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38101830 con 0x7fea3806b1e0 2024-05-09T08:29:22.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.604+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fea2402a070 con 0x7fea3806b1e0 2024-05-09T08:29:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.605+0000 7fea36000700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 0x7fea140a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.606+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fea240d0660 con 0x7fea3806b1e0 2024-05-09T08:29:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.606+0000 7fea3ce00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 0x7fea140a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.606+0000 7fea3ce00700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 0x7fea140a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009d60 tx=0x7fea1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.608+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fea24013070 con 0x7fea3806b1e0 2024-05-09T08:29:22.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.830+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea3819bfe0 con 0x7fea140a2d30 2024-05-09T08:29:22.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.839+0000 7fea36000700 1 -- 172.21.15.133:0/2876451261 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258933 (secure 0 0 0) 0x7fea3819bfe0 con 0x7fea140a2d30 2024-05-09T08:29:22.843 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 msgr2=0x7fea140a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 0x7fea140a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009d60 tx=0x7fea1c009260 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 msgr2=0x7fea381a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 0x7fea381a0f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea240059d0 tx=0x7fea2400bab0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 shutdown_connections 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea3810edc0 0x7fea3819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea3806a8c0 0x7fea381a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fea140a2d30 0x7fea140a51f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 --2- 172.21.15.133:0/2876451261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea3806b1e0 0x7fea381a0f90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.844+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 >> 172.21.15.133:0/2876451261 conn(0x7fea38078190 msgr2=0x7fea38110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.845+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 shutdown_connections 2024-05-09T08:29:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:22.845+0000 7fea3f800700 1 -- 172.21.15.133:0/2876451261 wait complete. 2024-05-09T08:29:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-05-09T08:29:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:29:23.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:29:23.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:29:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:29:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:29:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:29:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:29:23.796 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-05-09T08:29:22.695006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654068,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":627,"num_read_kb":427,"num_write":338,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373364,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568460,"statfs":{"total":1151940427776,"available":1151558103040,"internally_reserved":0,"allocated":11931648,"data_stored":7472100,"data_compressed":112701,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006021"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839179+0000","last_change":"2024-05-09T08:27:55.828188+0000","last_active":"2024-05-09T08:28:57.839179+0000","last_peered":"2024-05-09T08:28:57.839179+0000","last_clean":"2024-05-09T08:28:57.839179+0000","last_became_active":"2024-05-09T08:27:55.827858+0000","last_became_peered":"2024-05-09T08:27:55.827858+0000","last_unstale":"2024-05-09T08:28:57.839179+0000","last_undegraded":"2024-05-09T08:28:57.839179+0000","last_fullsized":"2024-05-09T08:28:57.839179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:19.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833309+0000","last_change":"2024-05-09T08:27:55.821504+0000","last_active":"2024-05-09T08:28:57.833309+0000","last_peered":"2024-05-09T08:28:57.833309+0000","last_clean":"2024-05-09T08:28:57.833309+0000","last_became_active":"2024-05-09T08:27:55.818102+0000","last_became_peered":"2024-05-09T08:27:55.818102+0000","last_unstale":"2024-05-09T08:28:57.833309+0000","last_undegraded":"2024-05-09T08:28:57.833309+0000","last_fullsized":"2024-05-09T08:28:57.833309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:00.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833859+0000","last_change":"2024-05-09T08:27:55.817240+0000","last_active":"2024-05-09T08:28:57.833859+0000","last_peered":"2024-05-09T08:28:57.833859+0000","last_clean":"2024-05-09T08:28:57.833859+0000","last_became_active":"2024-05-09T08:27:55.816865+0000","last_became_peered":"2024-05-09T08:27:55.816865+0000","last_unstale":"2024-05-09T08:28:57.833859+0000","last_undegraded":"2024-05-09T08:28:57.833859+0000","last_fullsized":"2024-05-09T08:28:57.833859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:04.149531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940132+0000","last_change":"2024-05-09T08:27:55.805704+0000","last_active":"2024-05-09T08:28:57.940132+0000","last_peered":"2024-05-09T08:28:57.940132+0000","last_clean":"2024-05-09T08:28:57.940132+0000","last_became_active":"2024-05-09T08:27:55.805344+0000","last_became_peered":"2024-05-09T08:27:55.805344+0000","last_unstale":"2024-05-09T08:28:57.940132+0000","last_undegraded":"2024-05-09T08:28:57.940132+0000","last_fullsized":"2024-05-09T08:28:57.940132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:02.418060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842021+0000","last_change":"2024-05-09T08:27:55.821747+0000","last_active":"2024-05-09T08:28:57.842021+0000","last_peered":"2024-05-09T08:28:57.842021+0000","last_clean":"2024-05-09T08:28:57.842021+0000","last_became_active":"2024-05-09T08:27:55.821378+0000","last_became_peered":"2024-05-09T08:27:55.821378+0000","last_unstale":"2024-05-09T08:28:57.842021+0000","last_undegraded":"2024-05-09T08:28:57.842021+0000","last_fullsized":"2024-05-09T08:28:57.842021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:53.470723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825644+0000","last_change":"2024-05-09T08:27:55.812827+0000","last_active":"2024-05-09T08:28:57.825644+0000","last_peered":"2024-05-09T08:28:57.825644+0000","last_clean":"2024-05-09T08:28:57.825644+0000","last_became_active":"2024-05-09T08:27:55.812412+0000","last_became_peered":"2024-05-09T08:27:55.812412+0000","last_unstale":"2024-05-09T08:28:57.825644+0000","last_undegraded":"2024-05-09T08:28:57.825644+0000","last_fullsized":"2024-05-09T08:28:57.825644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:20.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838677+0000","last_change":"2024-05-09T08:27:55.811536+0000","last_active":"2024-05-09T08:28:57.838677+0000","last_peered":"2024-05-09T08:28:57.838677+0000","last_clean":"2024-05-09T08:28:57.838677+0000","last_became_active":"2024-05-09T08:27:55.811181+0000","last_became_peered":"2024-05-09T08:27:55.811181+0000","last_unstale":"2024-05-09T08:28:57.838677+0000","last_undegraded":"2024-05-09T08:28:57.838677+0000","last_fullsized":"2024-05-09T08:28:57.838677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.845386+0000","last_change":"2024-05-09T08:28:57.842336+0000","last_active":"2024-05-09T08:28:57.845386+0000","last_peered":"2024-05-09T08:28:57.845386+0000","last_clean":"2024-05-09T08:28:57.845386+0000","last_became_active":"2024-05-09T08:28:57.841559+0000","last_became_peered":"2024-05-09T08:28:57.841559+0000","last_unstale":"2024-05-09T08:28:57.845386+0000","last_undegraded":"2024-05-09T08:28:57.845386+0000","last_fullsized":"2024-05-09T08:28:57.845386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:34.208823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.941205+0000","last_change":"2024-05-09T08:27:55.802154+0000","last_active":"2024-05-09T08:28:57.941205+0000","last_peered":"2024-05-09T08:28:57.941205+0000","last_clean":"2024-05-09T08:28:57.941205+0000","last_became_active":"2024-05-09T08:27:55.801501+0000","last_became_peered":"2024-05-09T08:27:55.801501+0000","last_unstale":"2024-05-09T08:28:57.941205+0000","last_undegraded":"2024-05-09T08:28:57.941205+0000","last_fullsized":"2024-05-09T08:28:57.941205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:42.009521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841929+0000","last_change":"2024-05-09T08:28:57.841929+0000","last_active":"2024-05-09T08:28:57.841929+0000","last_peered":"2024-05-09T08:28:57.841929+0000","last_clean":"2024-05-09T08:28:57.841929+0000","last_became_active":"2024-05-09T08:28:57.841717+0000","last_became_peered":"2024-05-09T08:28:57.841717+0000","last_unstale":"2024-05-09T08:28:57.841929+0000","last_undegraded":"2024-05-09T08:28:57.841929+0000","last_fullsized":"2024-05-09T08:28:57.841929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:42.099158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838874+0000","last_change":"2024-05-09T08:27:55.806122+0000","last_active":"2024-05-09T08:28:57.838874+0000","last_peered":"2024-05-09T08:28:57.838874+0000","last_clean":"2024-05-09T08:28:57.838874+0000","last_became_active":"2024-05-09T08:27:55.803844+0000","last_became_peered":"2024-05-09T08:27:55.803844+0000","last_unstale":"2024-05-09T08:28:57.838874+0000","last_undegraded":"2024-05-09T08:28:57.838874+0000","last_fullsized":"2024-05-09T08:28:57.838874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:40.692633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940023+0000","last_change":"2024-05-09T08:27:55.810184+0000","last_active":"2024-05-09T08:28:57.940023+0000","last_peered":"2024-05-09T08:28:57.940023+0000","last_clean":"2024-05-09T08:28:57.940023+0000","last_became_active":"2024-05-09T08:27:55.809833+0000","last_became_peered":"2024-05-09T08:27:55.809833+0000","last_unstale":"2024-05-09T08:28:57.940023+0000","last_undegraded":"2024-05-09T08:28:57.940023+0000","last_fullsized":"2024-05-09T08:28:57.940023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:19:33.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835264+0000","last_change":"2024-05-09T08:27:55.804770+0000","last_active":"2024-05-09T08:28:57.835264+0000","last_peered":"2024-05-09T08:28:57.835264+0000","last_clean":"2024-05-09T08:28:57.835264+0000","last_became_active":"2024-05-09T08:27:55.803486+0000","last_became_peered":"2024-05-09T08:27:55.803486+0000","last_unstale":"2024-05-09T08:28:57.835264+0000","last_undegraded":"2024-05-09T08:28:57.835264+0000","last_fullsized":"2024-05-09T08:28:57.835264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:28.641874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.832338+0000","last_change":"2024-05-09T08:27:55.826032+0000","last_active":"2024-05-09T08:28:57.832338+0000","last_peered":"2024-05-09T08:28:57.832338+0000","last_clean":"2024-05-09T08:28:57.832338+0000","last_became_active":"2024-05-09T08:27:55.825510+0000","last_became_peered":"2024-05-09T08:27:55.825510+0000","last_unstale":"2024-05-09T08:28:57.832338+0000","last_undegraded":"2024-05-09T08:28:57.832338+0000","last_fullsized":"2024-05-09T08:28:57.832338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:25.691361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.831679+0000","last_change":"2024-05-09T08:27:55.816178+0000","last_active":"2024-05-09T08:28:57.831679+0000","last_peered":"2024-05-09T08:28:57.831679+0000","last_clean":"2024-05-09T08:28:57.831679+0000","last_became_active":"2024-05-09T08:27:55.814592+0000","last_became_peered":"2024-05-09T08:27:55.814592+0000","last_unstale":"2024-05-09T08:28:57.831679+0000","last_undegraded":"2024-05-09T08:28:57.831679+0000","last_fullsized":"2024-05-09T08:28:57.831679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:06.632314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940772+0000","last_change":"2024-05-09T08:27:55.827914+0000","last_active":"2024-05-09T08:28:57.940772+0000","last_peered":"2024-05-09T08:28:57.940772+0000","last_clean":"2024-05-09T08:28:57.940772+0000","last_became_active":"2024-05-09T08:27:55.827555+0000","last_became_peered":"2024-05-09T08:27:55.827555+0000","last_unstale":"2024-05-09T08:28:57.940772+0000","last_undegraded":"2024-05-09T08:28:57.940772+0000","last_fullsized":"2024-05-09T08:28:57.940772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:37.097461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839443+0000","last_change":"2024-05-09T08:27:55.828072+0000","last_active":"2024-05-09T08:28:57.839443+0000","last_peered":"2024-05-09T08:28:57.839443+0000","last_clean":"2024-05-09T08:28:57.839443+0000","last_became_active":"2024-05-09T08:27:55.827727+0000","last_became_peered":"2024-05-09T08:27:55.827727+0000","last_unstale":"2024-05-09T08:28:57.839443+0000","last_undegraded":"2024-05-09T08:28:57.839443+0000","last_fullsized":"2024-05-09T08:28:57.839443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:13.761560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.829550+0000","last_change":"2024-05-09T08:27:55.812506+0000","last_active":"2024-05-09T08:28:57.829550+0000","last_peered":"2024-05-09T08:28:57.829550+0000","last_clean":"2024-05-09T08:28:57.829550+0000","last_became_active":"2024-05-09T08:27:55.812071+0000","last_became_peered":"2024-05-09T08:27:55.812071+0000","last_unstale":"2024-05-09T08:28:57.829550+0000","last_undegraded":"2024-05-09T08:28:57.829550+0000","last_fullsized":"2024-05-09T08:28:57.829550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:53.318140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841325+0000","last_change":"2024-05-09T08:27:55.817660+0000","last_active":"2024-05-09T08:28:57.841325+0000","last_peered":"2024-05-09T08:28:57.841325+0000","last_clean":"2024-05-09T08:28:57.841325+0000","last_became_active":"2024-05-09T08:27:55.817357+0000","last_became_peered":"2024-05-09T08:27:55.817357+0000","last_unstale":"2024-05-09T08:28:57.841325+0000","last_undegraded":"2024-05-09T08:28:57.841325+0000","last_fullsized":"2024-05-09T08:28:57.841325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:05.489063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.830096+0000","last_change":"2024-05-09T08:27:54.784177+0000","last_active":"2024-05-09T08:28:57.830096+0000","last_peered":"2024-05-09T08:28:57.830096+0000","last_clean":"2024-05-09T08:28:57.830096+0000","last_became_active":"2024-05-09T08:27:54.780982+0000","last_became_peered":"2024-05-09T08:27:54.780982+0000","last_unstale":"2024-05-09T08:28:57.830096+0000","last_undegraded":"2024-05-09T08:28:57.830096+0000","last_fullsized":"2024-05-09T08:28:57.830096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:17.668694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844135+0000","last_change":"2024-05-09T08:27:55.825495+0000","last_active":"2024-05-09T08:28:57.844135+0000","last_peered":"2024-05-09T08:28:57.844135+0000","last_clean":"2024-05-09T08:28:57.844135+0000","last_became_active":"2024-05-09T08:27:55.821680+0000","last_became_peered":"2024-05-09T08:27:55.821680+0000","last_unstale":"2024-05-09T08:28:57.844135+0000","last_undegraded":"2024-05-09T08:28:57.844135+0000","last_fullsized":"2024-05-09T08:28:57.844135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:06.128200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844264+0000","last_change":"2024-05-09T08:27:54.791362+0000","last_active":"2024-05-09T08:28:57.844264+0000","last_peered":"2024-05-09T08:28:57.844264+0000","last_clean":"2024-05-09T08:28:57.844264+0000","last_became_active":"2024-05-09T08:27:54.791050+0000","last_became_peered":"2024-05-09T08:27:54.791050+0000","last_unstale":"2024-05-09T08:28:57.844264+0000","last_undegraded":"2024-05-09T08:28:57.844264+0000","last_fullsized":"2024-05-09T08:28:57.844264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:11.821907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835743+0000","last_change":"2024-05-09T08:27:55.820278+0000","last_active":"2024-05-09T08:28:57.835743+0000","last_peered":"2024-05-09T08:28:57.835743+0000","last_clean":"2024-05-09T08:28:57.835743+0000","last_became_active":"2024-05-09T08:27:55.819861+0000","last_became_peered":"2024-05-09T08:27:55.819861+0000","last_unstale":"2024-05-09T08:28:57.835743+0000","last_undegraded":"2024-05-09T08:28:57.835743+0000","last_fullsized":"2024-05-09T08:28:57.835743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:29.541259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843371+0000","last_change":"2024-05-09T08:27:54.771453+0000","last_active":"2024-05-09T08:28:57.843371+0000","last_peered":"2024-05-09T08:28:57.843371+0000","last_clean":"2024-05-09T08:28:57.843371+0000","last_became_active":"2024-05-09T08:27:54.770746+0000","last_became_peered":"2024-05-09T08:27:54.770746+0000","last_unstale":"2024-05-09T08:28:57.843371+0000","last_undegraded":"2024-05-09T08:28:57.843371+0000","last_fullsized":"2024-05-09T08:28:57.843371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:18.823562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940718+0000","last_change":"2024-05-09T08:27:55.803341+0000","last_active":"2024-05-09T08:28:57.940718+0000","last_peered":"2024-05-09T08:28:57.940718+0000","last_clean":"2024-05-09T08:28:57.940718+0000","last_became_active":"2024-05-09T08:27:55.802945+0000","last_became_peered":"2024-05-09T08:27:55.802945+0000","last_unstale":"2024-05-09T08:28:57.940718+0000","last_undegraded":"2024-05-09T08:28:57.940718+0000","last_fullsized":"2024-05-09T08:28:57.940718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:24.346139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843074+0000","last_change":"2024-05-09T08:27:54.769140+0000","last_active":"2024-05-09T08:28:57.843074+0000","last_peered":"2024-05-09T08:28:57.843074+0000","last_clean":"2024-05-09T08:28:57.843074+0000","last_became_active":"2024-05-09T08:27:54.768615+0000","last_became_peered":"2024-05-09T08:27:54.768615+0000","last_unstale":"2024-05-09T08:28:57.843074+0000","last_undegraded":"2024-05-09T08:28:57.843074+0000","last_fullsized":"2024-05-09T08:28:57.843074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:10.016070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838835+0000","last_change":"2024-05-09T08:27:55.806385+0000","last_active":"2024-05-09T08:28:57.838835+0000","last_peered":"2024-05-09T08:28:57.838835+0000","last_clean":"2024-05-09T08:28:57.838835+0000","last_became_active":"2024-05-09T08:27:55.806031+0000","last_became_peered":"2024-05-09T08:27:55.806031+0000","last_unstale":"2024-05-09T08:28:57.838835+0000","last_undegraded":"2024-05-09T08:28:57.838835+0000","last_fullsized":"2024-05-09T08:28:57.838835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:17.326277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:17.395393+0000","last_change":"2024-05-09T08:27:54.769148+0000","last_active":"2024-05-09T08:29:17.395393+0000","last_peered":"2024-05-09T08:29:17.395393+0000","last_clean":"2024-05-09T08:29:17.395393+0000","last_became_active":"2024-05-09T08:27:54.768814+0000","last_became_peered":"2024-05-09T08:27:54.768814+0000","last_unstale":"2024-05-09T08:29:17.395393+0000","last_undegraded":"2024-05-09T08:29:17.395393+0000","last_fullsized":"2024-05-09T08:29:17.395393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:08.276294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842085+0000","last_change":"2024-05-09T08:27:55.812884+0000","last_active":"2024-05-09T08:28:57.842085+0000","last_peered":"2024-05-09T08:28:57.842085+0000","last_clean":"2024-05-09T08:28:57.842085+0000","last_became_active":"2024-05-09T08:27:55.812515+0000","last_became_peered":"2024-05-09T08:27:55.812515+0000","last_unstale":"2024-05-09T08:28:57.842085+0000","last_undegraded":"2024-05-09T08:28:57.842085+0000","last_fullsized":"2024-05-09T08:28:57.842085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:59.211293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825569+0000","last_change":"2024-05-09T08:27:54.764806+0000","last_active":"2024-05-09T08:28:57.825569+0000","last_peered":"2024-05-09T08:28:57.825569+0000","last_clean":"2024-05-09T08:28:57.825569+0000","last_became_active":"2024-05-09T08:27:54.764325+0000","last_became_peered":"2024-05-09T08:27:54.764325+0000","last_unstale":"2024-05-09T08:28:57.825569+0000","last_undegraded":"2024-05-09T08:28:57.825569+0000","last_fullsized":"2024-05-09T08:28:57.825569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:11.820240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838697+0000","last_change":"2024-05-09T08:27:55.822481+0000","last_active":"2024-05-09T08:28:57.838697+0000","last_peered":"2024-05-09T08:28:57.838697+0000","last_clean":"2024-05-09T08:28:57.838697+0000","last_became_active":"2024-05-09T08:27:55.821180+0000","last_became_peered":"2024-05-09T08:27:55.821180+0000","last_unstale":"2024-05-09T08:28:57.838697+0000","last_undegraded":"2024-05-09T08:28:57.838697+0000","last_fullsized":"2024-05-09T08:28:57.838697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:33:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834771+0000","last_change":"2024-05-09T08:27:54.771263+0000","last_active":"2024-05-09T08:28:57.834771+0000","last_peered":"2024-05-09T08:28:57.834771+0000","last_clean":"2024-05-09T08:28:57.834771+0000","last_became_active":"2024-05-09T08:27:54.770719+0000","last_became_peered":"2024-05-09T08:27:54.770719+0000","last_unstale":"2024-05-09T08:28:57.834771+0000","last_undegraded":"2024-05-09T08:28:57.834771+0000","last_fullsized":"2024-05-09T08:28:57.834771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:08.725081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842207+0000","last_change":"2024-05-09T08:27:55.821493+0000","last_active":"2024-05-09T08:28:57.842207+0000","last_peered":"2024-05-09T08:28:57.842207+0000","last_clean":"2024-05-09T08:28:57.842207+0000","last_became_active":"2024-05-09T08:27:55.818585+0000","last_became_peered":"2024-05-09T08:27:55.818585+0000","last_unstale":"2024-05-09T08:28:57.842207+0000","last_undegraded":"2024-05-09T08:28:57.842207+0000","last_fullsized":"2024-05-09T08:28:57.842207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:56.319691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839532+0000","last_change":"2024-05-09T08:27:54.769856+0000","last_active":"2024-05-09T08:28:57.839532+0000","last_peered":"2024-05-09T08:28:57.839532+0000","last_clean":"2024-05-09T08:28:57.839532+0000","last_became_active":"2024-05-09T08:27:54.769548+0000","last_became_peered":"2024-05-09T08:27:54.769548+0000","last_unstale":"2024-05-09T08:28:57.839532+0000","last_undegraded":"2024-05-09T08:28:57.839532+0000","last_fullsized":"2024-05-09T08:28:57.839532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:57.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.828624+0000","last_change":"2024-05-09T08:27:55.822063+0000","last_active":"2024-05-09T08:28:57.828624+0000","last_peered":"2024-05-09T08:28:57.828624+0000","last_clean":"2024-05-09T08:28:57.828624+0000","last_became_active":"2024-05-09T08:27:55.821624+0000","last_became_peered":"2024-05-09T08:27:55.821624+0000","last_unstale":"2024-05-09T08:28:57.828624+0000","last_undegraded":"2024-05-09T08:28:57.828624+0000","last_fullsized":"2024-05-09T08:28:57.828624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:14.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":103,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:20.398797+0000","last_change":"2024-05-09T08:27:54.767639+0000","last_active":"2024-05-09T08:29:20.398797+0000","last_peered":"2024-05-09T08:29:20.398797+0000","last_clean":"2024-05-09T08:29:20.398797+0000","last_became_active":"2024-05-09T08:27:54.767284+0000","last_became_peered":"2024-05-09T08:27:54.767284+0000","last_unstale":"2024-05-09T08:29:20.398797+0000","last_undegraded":"2024-05-09T08:29:20.398797+0000","last_fullsized":"2024-05-09T08:29:20.398797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:07.110319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940992+0000","last_change":"2024-05-09T08:27:55.817050+0000","last_active":"2024-05-09T08:28:57.940992+0000","last_peered":"2024-05-09T08:28:57.940992+0000","last_clean":"2024-05-09T08:28:57.940992+0000","last_became_active":"2024-05-09T08:27:55.816655+0000","last_became_peered":"2024-05-09T08:27:55.816655+0000","last_unstale":"2024-05-09T08:28:57.940992+0000","last_undegraded":"2024-05-09T08:28:57.940992+0000","last_fullsized":"2024-05-09T08:28:57.940992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:06.132482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838870+0000","last_change":"2024-05-09T08:27:54.770449+0000","last_active":"2024-05-09T08:28:57.838870+0000","last_peered":"2024-05-09T08:28:57.838870+0000","last_clean":"2024-05-09T08:28:57.838870+0000","last_became_active":"2024-05-09T08:27:54.769382+0000","last_became_peered":"2024-05-09T08:27:54.769382+0000","last_unstale":"2024-05-09T08:28:57.838870+0000","last_undegraded":"2024-05-09T08:28:57.838870+0000","last_fullsized":"2024-05-09T08:28:57.838870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:47.108749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834387+0000","last_change":"2024-05-09T08:27:55.831416+0000","last_active":"2024-05-09T08:28:57.834387+0000","last_peered":"2024-05-09T08:28:57.834387+0000","last_clean":"2024-05-09T08:28:57.834387+0000","last_became_active":"2024-05-09T08:27:55.831011+0000","last_became_peered":"2024-05-09T08:27:55.831011+0000","last_unstale":"2024-05-09T08:28:57.834387+0000","last_undegraded":"2024-05-09T08:28:57.834387+0000","last_fullsized":"2024-05-09T08:28:57.834387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:23.820425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838929+0000","last_change":"2024-05-09T08:27:54.776737+0000","last_active":"2024-05-09T08:28:57.838929+0000","last_peered":"2024-05-09T08:28:57.838929+0000","last_clean":"2024-05-09T08:28:57.838929+0000","last_became_active":"2024-05-09T08:27:54.776402+0000","last_became_peered":"2024-05-09T08:27:54.776402+0000","last_unstale":"2024-05-09T08:28:57.838929+0000","last_undegraded":"2024-05-09T08:28:57.838929+0000","last_fullsized":"2024-05-09T08:28:57.838929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:38.135159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940931+0000","last_change":"2024-05-09T08:27:55.810092+0000","last_active":"2024-05-09T08:28:57.940931+0000","last_peered":"2024-05-09T08:28:57.940931+0000","last_clean":"2024-05-09T08:28:57.940931+0000","last_became_active":"2024-05-09T08:27:55.809738+0000","last_became_peered":"2024-05-09T08:27:55.809738+0000","last_unstale":"2024-05-09T08:28:57.940931+0000","last_undegraded":"2024-05-09T08:28:57.940931+0000","last_fullsized":"2024-05-09T08:28:57.940931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:34.094014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":215,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:21.405562+0000","last_change":"2024-05-09T08:27:54.769608+0000","last_active":"2024-05-09T08:29:21.405562+0000","last_peered":"2024-05-09T08:29:21.405562+0000","last_clean":"2024-05-09T08:29:21.405562+0000","last_became_active":"2024-05-09T08:27:54.769286+0000","last_became_peered":"2024-05-09T08:27:54.769286+0000","last_unstale":"2024-05-09T08:29:21.405562+0000","last_undegraded":"2024-05-09T08:29:21.405562+0000","last_fullsized":"2024-05-09T08:29:21.405562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:40.620602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":169,"num_read_kb":142,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842882+0000","last_change":"2024-05-09T08:27:55.800489+0000","last_active":"2024-05-09T08:28:57.842882+0000","last_peered":"2024-05-09T08:28:57.842882+0000","last_clean":"2024-05-09T08:28:57.842882+0000","last_became_active":"2024-05-09T08:27:55.800145+0000","last_became_peered":"2024-05-09T08:27:55.800145+0000","last_unstale":"2024-05-09T08:28:57.842882+0000","last_undegraded":"2024-05-09T08:28:57.842882+0000","last_fullsized":"2024-05-09T08:28:57.842882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:47.877630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841014+0000","last_change":"2024-05-09T08:27:54.779179+0000","last_active":"2024-05-09T08:28:57.841014+0000","last_peered":"2024-05-09T08:28:57.841014+0000","last_clean":"2024-05-09T08:28:57.841014+0000","last_became_active":"2024-05-09T08:27:54.778793+0000","last_became_peered":"2024-05-09T08:27:54.778793+0000","last_unstale":"2024-05-09T08:28:57.841014+0000","last_undegraded":"2024-05-09T08:28:57.841014+0000","last_fullsized":"2024-05-09T08:28:57.841014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:56:42.414959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835498+0000","last_change":"2024-05-09T08:27:55.830500+0000","last_active":"2024-05-09T08:28:57.835498+0000","last_peered":"2024-05-09T08:28:57.835498+0000","last_clean":"2024-05-09T08:28:57.835498+0000","last_became_active":"2024-05-09T08:27:55.830044+0000","last_became_peered":"2024-05-09T08:27:55.830044+0000","last_unstale":"2024-05-09T08:28:57.835498+0000","last_undegraded":"2024-05-09T08:28:57.835498+0000","last_fullsized":"2024-05-09T08:28:57.835498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:15.616303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838889+0000","last_change":"2024-05-09T08:27:54.786607+0000","last_active":"2024-05-09T08:28:57.838889+0000","last_peered":"2024-05-09T08:28:57.838889+0000","last_clean":"2024-05-09T08:28:57.838889+0000","last_became_active":"2024-05-09T08:27:54.786286+0000","last_became_peered":"2024-05-09T08:27:54.786286+0000","last_unstale":"2024-05-09T08:28:57.838889+0000","last_undegraded":"2024-05-09T08:28:57.838889+0000","last_fullsized":"2024-05-09T08:28:57.838889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.833237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838921+0000","last_change":"2024-05-09T08:27:55.809022+0000","last_active":"2024-05-09T08:28:57.838921+0000","last_peered":"2024-05-09T08:28:57.838921+0000","last_clean":"2024-05-09T08:28:57.838921+0000","last_became_active":"2024-05-09T08:27:55.808666+0000","last_became_peered":"2024-05-09T08:27:55.808666+0000","last_unstale":"2024-05-09T08:28:57.838921+0000","last_undegraded":"2024-05-09T08:28:57.838921+0000","last_fullsized":"2024-05-09T08:28:57.838921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:18.339576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:17.445863+0000","last_change":"2024-05-09T08:27:54.769166+0000","last_active":"2024-05-09T08:29:17.445863+0000","last_peered":"2024-05-09T08:29:17.445863+0000","last_clean":"2024-05-09T08:29:17.445863+0000","last_became_active":"2024-05-09T08:27:54.768794+0000","last_became_peered":"2024-05-09T08:27:54.768794+0000","last_unstale":"2024-05-09T08:29:17.445863+0000","last_undegraded":"2024-05-09T08:29:17.445863+0000","last_fullsized":"2024-05-09T08:29:17.445863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.222388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838602+0000","last_change":"2024-05-09T08:27:55.806685+0000","last_active":"2024-05-09T08:28:57.838602+0000","last_peered":"2024-05-09T08:28:57.838602+0000","last_clean":"2024-05-09T08:28:57.838602+0000","last_became_active":"2024-05-09T08:27:55.804641+0000","last_became_peered":"2024-05-09T08:27:55.804641+0000","last_unstale":"2024-05-09T08:28:57.838602+0000","last_undegraded":"2024-05-09T08:28:57.838602+0000","last_fullsized":"2024-05-09T08:28:57.838602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:40.307764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842320+0000","last_change":"2024-05-09T08:27:54.782524+0000","last_active":"2024-05-09T08:28:57.842320+0000","last_peered":"2024-05-09T08:28:57.842320+0000","last_clean":"2024-05-09T08:28:57.842320+0000","last_became_active":"2024-05-09T08:27:54.782220+0000","last_became_peered":"2024-05-09T08:27:54.782220+0000","last_unstale":"2024-05-09T08:28:57.842320+0000","last_undegraded":"2024-05-09T08:28:57.842320+0000","last_fullsized":"2024-05-09T08:28:57.842320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:16:58.705682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838632+0000","last_change":"2024-05-09T08:27:55.806390+0000","last_active":"2024-05-09T08:28:57.838632+0000","last_peered":"2024-05-09T08:28:57.838632+0000","last_clean":"2024-05-09T08:28:57.838632+0000","last_became_active":"2024-05-09T08:27:55.806027+0000","last_became_peered":"2024-05-09T08:27:55.806027+0000","last_unstale":"2024-05-09T08:28:57.838632+0000","last_undegraded":"2024-05-09T08:28:57.838632+0000","last_fullsized":"2024-05-09T08:28:57.838632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:58.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842884+0000","last_change":"2024-05-09T08:27:54.766679+0000","last_active":"2024-05-09T08:28:57.842884+0000","last_peered":"2024-05-09T08:28:57.842884+0000","last_clean":"2024-05-09T08:28:57.842884+0000","last_became_active":"2024-05-09T08:27:54.766318+0000","last_became_peered":"2024-05-09T08:27:54.766318+0000","last_unstale":"2024-05-09T08:28:57.842884+0000","last_undegraded":"2024-05-09T08:28:57.842884+0000","last_fullsized":"2024-05-09T08:28:57.842884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:38.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838541+0000","last_change":"2024-05-09T08:27:55.804379+0000","last_active":"2024-05-09T08:28:57.838541+0000","last_peered":"2024-05-09T08:28:57.838541+0000","last_clean":"2024-05-09T08:28:57.838541+0000","last_became_active":"2024-05-09T08:27:55.804046+0000","last_became_peered":"2024-05-09T08:27:55.804046+0000","last_unstale":"2024-05-09T08:28:57.838541+0000","last_undegraded":"2024-05-09T08:28:57.838541+0000","last_fullsized":"2024-05-09T08:28:57.838541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:43.735487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.831056+0000","last_change":"2024-05-09T08:27:54.785336+0000","last_active":"2024-05-09T08:28:57.831056+0000","last_peered":"2024-05-09T08:28:57.831056+0000","last_clean":"2024-05-09T08:28:57.831056+0000","last_became_active":"2024-05-09T08:27:54.784581+0000","last_became_peered":"2024-05-09T08:27:54.784581+0000","last_unstale":"2024-05-09T08:28:57.831056+0000","last_undegraded":"2024-05-09T08:28:57.831056+0000","last_fullsized":"2024-05-09T08:28:57.831056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:40.071747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825920+0000","last_change":"2024-05-09T08:27:55.821512+0000","last_active":"2024-05-09T08:28:57.825920+0000","last_peered":"2024-05-09T08:28:57.825920+0000","last_clean":"2024-05-09T08:28:57.825920+0000","last_became_active":"2024-05-09T08:27:55.821162+0000","last_became_peered":"2024-05-09T08:27:55.821162+0000","last_unstale":"2024-05-09T08:28:57.825920+0000","last_undegraded":"2024-05-09T08:28:57.825920+0000","last_fullsized":"2024-05-09T08:28:57.825920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:47.377452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838776+0000","last_change":"2024-05-09T08:27:54.771528+0000","last_active":"2024-05-09T08:28:57.838776+0000","last_peered":"2024-05-09T08:28:57.838776+0000","last_clean":"2024-05-09T08:28:57.838776+0000","last_became_active":"2024-05-09T08:27:54.771133+0000","last_became_peered":"2024-05-09T08:27:54.771133+0000","last_unstale":"2024-05-09T08:28:57.838776+0000","last_undegraded":"2024-05-09T08:28:57.838776+0000","last_fullsized":"2024-05-09T08:28:57.838776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:22.948800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825795+0000","last_change":"2024-05-09T08:27:55.824954+0000","last_active":"2024-05-09T08:28:57.825795+0000","last_peered":"2024-05-09T08:28:57.825795+0000","last_clean":"2024-05-09T08:28:57.825795+0000","last_became_active":"2024-05-09T08:27:55.824578+0000","last_became_peered":"2024-05-09T08:27:55.824578+0000","last_unstale":"2024-05-09T08:28:57.825795+0000","last_undegraded":"2024-05-09T08:28:57.825795+0000","last_fullsized":"2024-05-09T08:28:57.825795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:35:22.790193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844413+0000","last_change":"2024-05-09T08:28:57.844413+0000","last_active":"2024-05-09T08:28:57.844413+0000","last_peered":"2024-05-09T08:28:57.844413+0000","last_clean":"2024-05-09T08:28:57.844413+0000","last_became_active":"2024-05-09T08:28:57.844057+0000","last_became_peered":"2024-05-09T08:28:57.844057+0000","last_unstale":"2024-05-09T08:28:57.844413+0000","last_undegraded":"2024-05-09T08:28:57.844413+0000","last_fullsized":"2024-05-09T08:28:57.844413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:55.603293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":101,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:17.444822+0000","last_change":"2024-05-09T08:27:54.780486+0000","last_active":"2024-05-09T08:29:17.444822+0000","last_peered":"2024-05-09T08:29:17.444822+0000","last_clean":"2024-05-09T08:29:17.444822+0000","last_became_active":"2024-05-09T08:27:54.777349+0000","last_became_peered":"2024-05-09T08:27:54.777349+0000","last_unstale":"2024-05-09T08:29:17.444822+0000","last_undegraded":"2024-05-09T08:29:17.444822+0000","last_fullsized":"2024-05-09T08:29:17.444822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:14.729741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844568+0000","last_change":"2024-05-09T08:27:55.825469+0000","last_active":"2024-05-09T08:28:57.844568+0000","last_peered":"2024-05-09T08:28:57.844568+0000","last_clean":"2024-05-09T08:28:57.844568+0000","last_became_active":"2024-05-09T08:27:55.825103+0000","last_became_peered":"2024-05-09T08:27:55.825103+0000","last_unstale":"2024-05-09T08:28:57.844568+0000","last_undegraded":"2024-05-09T08:28:57.844568+0000","last_fullsized":"2024-05-09T08:28:57.844568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:53.136741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.939761+0000","last_change":"2024-05-09T08:27:54.774194+0000","last_active":"2024-05-09T08:28:57.939761+0000","last_peered":"2024-05-09T08:28:57.939761+0000","last_clean":"2024-05-09T08:28:57.939761+0000","last_became_active":"2024-05-09T08:27:54.773678+0000","last_became_peered":"2024-05-09T08:27:54.773678+0000","last_unstale":"2024-05-09T08:28:57.939761+0000","last_undegraded":"2024-05-09T08:28:57.939761+0000","last_fullsized":"2024-05-09T08:28:57.939761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:23.199301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838704+0000","last_change":"2024-05-09T08:26:34.966221+0000","last_active":"2024-05-09T08:28:57.838704+0000","last_peered":"2024-05-09T08:28:57.838704+0000","last_clean":"2024-05-09T08:28:57.838704+0000","last_became_active":"2024-05-09T08:26:34.965800+0000","last_became_peered":"2024-05-09T08:26:34.965800+0000","last_unstale":"2024-05-09T08:28:57.838704+0000","last_undegraded":"2024-05-09T08:28:57.838704+0000","last_fullsized":"2024-05-09T08:28:57.838704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_clean_scrub_stamp":"2024-05-09T08:22:41.596226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:10.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834552+0000","last_change":"2024-05-09T08:27:55.805417+0000","last_active":"2024-05-09T08:28:57.834552+0000","last_peered":"2024-05-09T08:28:57.834552+0000","last_clean":"2024-05-09T08:28:57.834552+0000","last_became_active":"2024-05-09T08:27:55.805025+0000","last_became_peered":"2024-05-09T08:27:55.805025+0000","last_unstale":"2024-05-09T08:28:57.834552+0000","last_undegraded":"2024-05-09T08:28:57.834552+0000","last_fullsized":"2024-05-09T08:28:57.834552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:35.442848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839002+0000","last_change":"2024-05-09T08:27:54.769276+0000","last_active":"2024-05-09T08:28:57.839002+0000","last_peered":"2024-05-09T08:28:57.839002+0000","last_clean":"2024-05-09T08:28:57.839002+0000","last_became_active":"2024-05-09T08:27:54.768650+0000","last_became_peered":"2024-05-09T08:27:54.768650+0000","last_unstale":"2024-05-09T08:28:57.839002+0000","last_undegraded":"2024-05-09T08:28:57.839002+0000","last_fullsized":"2024-05-09T08:28:57.839002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:32.480610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842994+0000","last_change":"2024-05-09T08:27:54.782809+0000","last_active":"2024-05-09T08:28:57.842994+0000","last_peered":"2024-05-09T08:28:57.842994+0000","last_clean":"2024-05-09T08:28:57.842994+0000","last_became_active":"2024-05-09T08:27:54.782452+0000","last_became_peered":"2024-05-09T08:27:54.782452+0000","last_unstale":"2024-05-09T08:28:57.842994+0000","last_undegraded":"2024-05-09T08:28:57.842994+0000","last_fullsized":"2024-05-09T08:28:57.842994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:42.371573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835639+0000","last_change":"2024-05-09T08:27:55.814683+0000","last_active":"2024-05-09T08:28:57.835639+0000","last_peered":"2024-05-09T08:28:57.835639+0000","last_clean":"2024-05-09T08:28:57.835639+0000","last_became_active":"2024-05-09T08:27:55.813788+0000","last_became_peered":"2024-05-09T08:27:55.813788+0000","last_unstale":"2024-05-09T08:28:57.835639+0000","last_undegraded":"2024-05-09T08:28:57.835639+0000","last_fullsized":"2024-05-09T08:28:57.835639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:45.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.829381+0000","last_change":"2024-05-09T08:27:54.788393+0000","last_active":"2024-05-09T08:28:57.829381+0000","last_peered":"2024-05-09T08:28:57.829381+0000","last_clean":"2024-05-09T08:28:57.829381+0000","last_became_active":"2024-05-09T08:27:54.788031+0000","last_became_peered":"2024-05-09T08:27:54.788031+0000","last_unstale":"2024-05-09T08:28:57.829381+0000","last_undegraded":"2024-05-09T08:28:57.829381+0000","last_fullsized":"2024-05-09T08:28:57.829381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:26.451385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844858+0000","last_change":"2024-05-09T08:27:55.812391+0000","last_active":"2024-05-09T08:28:57.844858+0000","last_peered":"2024-05-09T08:28:57.844858+0000","last_clean":"2024-05-09T08:28:57.844858+0000","last_became_active":"2024-05-09T08:27:55.811954+0000","last_became_peered":"2024-05-09T08:27:55.811954+0000","last_unstale":"2024-05-09T08:28:57.844858+0000","last_undegraded":"2024-05-09T08:28:57.844858+0000","last_fullsized":"2024-05-09T08:28:57.844858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:38.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940042+0000","last_change":"2024-05-09T08:27:54.775693+0000","last_active":"2024-05-09T08:28:57.940042+0000","last_peered":"2024-05-09T08:28:57.940042+0000","last_clean":"2024-05-09T08:28:57.940042+0000","last_became_active":"2024-05-09T08:27:54.775198+0000","last_became_peered":"2024-05-09T08:27:54.775198+0000","last_unstale":"2024-05-09T08:28:57.940042+0000","last_undegraded":"2024-05-09T08:28:57.940042+0000","last_fullsized":"2024-05-09T08:28:57.940042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:21:18.672744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833562+0000","last_change":"2024-05-09T08:27:55.822832+0000","last_active":"2024-05-09T08:28:57.833562+0000","last_peered":"2024-05-09T08:28:57.833562+0000","last_clean":"2024-05-09T08:28:57.833562+0000","last_became_active":"2024-05-09T08:27:55.822391+0000","last_became_peered":"2024-05-09T08:27:55.822391+0000","last_unstale":"2024-05-09T08:28:57.833562+0000","last_undegraded":"2024-05-09T08:28:57.833562+0000","last_fullsized":"2024-05-09T08:28:57.833562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:00.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940621+0000","last_change":"2024-05-09T08:27:54.791027+0000","last_active":"2024-05-09T08:28:57.940621+0000","last_peered":"2024-05-09T08:28:57.940621+0000","last_clean":"2024-05-09T08:28:57.940621+0000","last_became_active":"2024-05-09T08:27:54.790726+0000","last_became_peered":"2024-05-09T08:27:54.790726+0000","last_unstale":"2024-05-09T08:28:57.940621+0000","last_undegraded":"2024-05-09T08:28:57.940621+0000","last_fullsized":"2024-05-09T08:28:57.940621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:37.795710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841994+0000","last_change":"2024-05-09T08:27:55.799349+0000","last_active":"2024-05-09T08:28:57.841994+0000","last_peered":"2024-05-09T08:28:57.841994+0000","last_clean":"2024-05-09T08:28:57.841994+0000","last_became_active":"2024-05-09T08:27:55.798931+0000","last_became_peered":"2024-05-09T08:27:55.798931+0000","last_unstale":"2024-05-09T08:28:57.841994+0000","last_undegraded":"2024-05-09T08:28:57.841994+0000","last_fullsized":"2024-05-09T08:28:57.841994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:15.824884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843389+0000","last_change":"2024-05-09T08:27:54.792088+0000","last_active":"2024-05-09T08:28:57.843389+0000","last_peered":"2024-05-09T08:28:57.843389+0000","last_clean":"2024-05-09T08:28:57.843389+0000","last_became_active":"2024-05-09T08:27:54.791526+0000","last_became_peered":"2024-05-09T08:27:54.791526+0000","last_unstale":"2024-05-09T08:28:57.843389+0000","last_undegraded":"2024-05-09T08:28:57.843389+0000","last_fullsized":"2024-05-09T08:28:57.843389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:11.494625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838345+0000","last_change":"2024-05-09T08:27:55.823258+0000","last_active":"2024-05-09T08:28:57.838345+0000","last_peered":"2024-05-09T08:28:57.838345+0000","last_clean":"2024-05-09T08:28:57.838345+0000","last_became_active":"2024-05-09T08:27:55.822923+0000","last_became_peered":"2024-05-09T08:27:55.822923+0000","last_unstale":"2024-05-09T08:28:57.838345+0000","last_undegraded":"2024-05-09T08:28:57.838345+0000","last_fullsized":"2024-05-09T08:28:57.838345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:17.481997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.849289+0000","last_change":"2024-05-09T08:28:57.848915+0000","last_active":"2024-05-09T08:28:57.849289+0000","last_peered":"2024-05-09T08:28:57.849289+0000","last_clean":"2024-05-09T08:28:57.849289+0000","last_became_active":"2024-05-09T08:28:57.844301+0000","last_became_peered":"2024-05-09T08:28:57.844301+0000","last_unstale":"2024-05-09T08:28:57.849289+0000","last_undegraded":"2024-05-09T08:28:57.849289+0000","last_fullsized":"2024-05-09T08:28:57.849289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:59.620437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843102+0000","last_change":"2024-05-09T08:27:55.829385+0000","last_active":"2024-05-09T08:28:57.843102+0000","last_peered":"2024-05-09T08:28:57.843102+0000","last_clean":"2024-05-09T08:28:57.843102+0000","last_became_active":"2024-05-09T08:27:55.829045+0000","last_became_peered":"2024-05-09T08:27:55.829045+0000","last_unstale":"2024-05-09T08:28:57.843102+0000","last_undegraded":"2024-05-09T08:28:57.843102+0000","last_fullsized":"2024-05-09T08:28:57.843102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.171868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.847090+0000","last_change":"2024-05-09T08:28:57.846755+0000","last_active":"2024-05-09T08:28:57.847090+0000","last_peered":"2024-05-09T08:28:57.847090+0000","last_clean":"2024-05-09T08:28:57.847090+0000","last_became_active":"2024-05-09T08:28:57.841834+0000","last_became_peered":"2024-05-09T08:28:57.841834+0000","last_unstale":"2024-05-09T08:28:57.847090+0000","last_undegraded":"2024-05-09T08:28:57.847090+0000","last_fullsized":"2024-05-09T08:28:57.847090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:08.123561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838949+0000","last_change":"2024-05-09T08:27:55.804210+0000","last_active":"2024-05-09T08:28:57.838949+0000","last_peered":"2024-05-09T08:28:57.838949+0000","last_clean":"2024-05-09T08:28:57.838949+0000","last_became_active":"2024-05-09T08:27:55.803870+0000","last_became_peered":"2024-05-09T08:27:55.803870+0000","last_unstale":"2024-05-09T08:28:57.838949+0000","last_undegraded":"2024-05-09T08:28:57.838949+0000","last_fullsized":"2024-05-09T08:28:57.838949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:46.593084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841120+0000","last_change":"2024-05-09T08:27:54.775110+0000","last_active":"2024-05-09T08:28:57.841120+0000","last_peered":"2024-05-09T08:28:57.841120+0000","last_clean":"2024-05-09T08:28:57.841120+0000","last_became_active":"2024-05-09T08:27:54.774812+0000","last_became_peered":"2024-05-09T08:27:54.774812+0000","last_unstale":"2024-05-09T08:28:57.841120+0000","last_undegraded":"2024-05-09T08:28:57.841120+0000","last_fullsized":"2024-05-09T08:28:57.841120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:51.955284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838938+0000","last_change":"2024-05-09T08:27:55.804766+0000","last_active":"2024-05-09T08:28:57.838938+0000","last_peered":"2024-05-09T08:28:57.838938+0000","last_clean":"2024-05-09T08:28:57.838938+0000","last_became_active":"2024-05-09T08:27:55.803711+0000","last_became_peered":"2024-05-09T08:27:55.803711+0000","last_unstale":"2024-05-09T08:28:57.838938+0000","last_undegraded":"2024-05-09T08:28:57.838938+0000","last_fullsized":"2024-05-09T08:28:57.838938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:46.683112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838726+0000","last_change":"2024-05-09T08:27:54.786594+0000","last_active":"2024-05-09T08:28:57.838726+0000","last_peered":"2024-05-09T08:28:57.838726+0000","last_clean":"2024-05-09T08:28:57.838726+0000","last_became_active":"2024-05-09T08:27:54.786329+0000","last_became_peered":"2024-05-09T08:27:54.786329+0000","last_unstale":"2024-05-09T08:28:57.838726+0000","last_undegraded":"2024-05-09T08:28:57.838726+0000","last_fullsized":"2024-05-09T08:28:57.838726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:09:43.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.849973+0000","last_change":"2024-05-09T08:28:57.849973+0000","last_active":"2024-05-09T08:28:57.849973+0000","last_peered":"2024-05-09T08:28:57.849973+0000","last_clean":"2024-05-09T08:28:57.849973+0000","last_became_active":"2024-05-09T08:28:57.844256+0000","last_became_peered":"2024-05-09T08:28:57.844256+0000","last_unstale":"2024-05-09T08:28:57.849973+0000","last_undegraded":"2024-05-09T08:28:57.849973+0000","last_fullsized":"2024-05-09T08:28:57.849973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:56.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835083+0000","last_change":"2024-05-09T08:27:54.766960+0000","last_active":"2024-05-09T08:28:57.835083+0000","last_peered":"2024-05-09T08:28:57.835083+0000","last_clean":"2024-05-09T08:28:57.835083+0000","last_became_active":"2024-05-09T08:27:54.765693+0000","last_became_peered":"2024-05-09T08:27:54.765693+0000","last_unstale":"2024-05-09T08:28:57.835083+0000","last_undegraded":"2024-05-09T08:28:57.835083+0000","last_fullsized":"2024-05-09T08:28:57.835083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:29:50.080788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838822+0000","last_change":"2024-05-09T08:27:55.806802+0000","last_active":"2024-05-09T08:28:57.838822+0000","last_peered":"2024-05-09T08:28:57.838822+0000","last_clean":"2024-05-09T08:28:57.838822+0000","last_became_active":"2024-05-09T08:27:55.806421+0000","last_became_peered":"2024-05-09T08:27:55.806421+0000","last_unstale":"2024-05-09T08:28:57.838822+0000","last_undegraded":"2024-05-09T08:28:57.838822+0000","last_fullsized":"2024-05-09T08:28:57.838822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:45.673043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838373+0000","last_change":"2024-05-09T08:27:55.799360+0000","last_active":"2024-05-09T08:28:57.838373+0000","last_peered":"2024-05-09T08:28:57.838373+0000","last_clean":"2024-05-09T08:28:57.838373+0000","last_became_active":"2024-05-09T08:27:55.799018+0000","last_became_peered":"2024-05-09T08:27:55.799018+0000","last_unstale":"2024-05-09T08:28:57.838373+0000","last_undegraded":"2024-05-09T08:28:57.838373+0000","last_fullsized":"2024-05-09T08:28:57.838373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:54.767546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.840915+0000","last_change":"2024-05-09T08:27:55.803385+0000","last_active":"2024-05-09T08:28:57.840915+0000","last_peered":"2024-05-09T08:28:57.840915+0000","last_clean":"2024-05-09T08:28:57.840915+0000","last_became_active":"2024-05-09T08:27:55.803031+0000","last_became_peered":"2024-05-09T08:27:55.803031+0000","last_unstale":"2024-05-09T08:28:57.840915+0000","last_undegraded":"2024-05-09T08:28:57.840915+0000","last_fullsized":"2024-05-09T08:28:57.840915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:47.750225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841167+0000","last_change":"2024-05-09T08:27:55.825634+0000","last_active":"2024-05-09T08:28:57.841167+0000","last_peered":"2024-05-09T08:28:57.841167+0000","last_clean":"2024-05-09T08:28:57.841167+0000","last_became_active":"2024-05-09T08:27:55.825330+0000","last_became_peered":"2024-05-09T08:27:55.825330+0000","last_unstale":"2024-05-09T08:28:57.841167+0000","last_undegraded":"2024-05-09T08:28:57.841167+0000","last_fullsized":"2024-05-09T08:28:57.841167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:21:18.166324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.832666+0000","last_change":"2024-05-09T08:27:55.818372+0000","last_active":"2024-05-09T08:28:57.832666+0000","last_peered":"2024-05-09T08:28:57.832666+0000","last_clean":"2024-05-09T08:28:57.832666+0000","last_became_active":"2024-05-09T08:27:55.817982+0000","last_became_peered":"2024-05-09T08:27:55.817982+0000","last_unstale":"2024-05-09T08:28:57.832666+0000","last_undegraded":"2024-05-09T08:28:57.832666+0000","last_fullsized":"2024-05-09T08:28:57.832666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:16.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.836616+0000","last_change":"2024-05-09T08:27:55.820509+0000","last_active":"2024-05-09T08:28:57.836616+0000","last_peered":"2024-05-09T08:28:57.836616+0000","last_clean":"2024-05-09T08:28:57.836616+0000","last_became_active":"2024-05-09T08:27:55.820135+0000","last_became_peered":"2024-05-09T08:27:55.820135+0000","last_unstale":"2024-05-09T08:28:57.836616+0000","last_undegraded":"2024-05-09T08:28:57.836616+0000","last_fullsized":"2024-05-09T08:28:57.836616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:28.691640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839350+0000","last_change":"2024-05-09T08:27:55.801038+0000","last_active":"2024-05-09T08:28:57.839350+0000","last_peered":"2024-05-09T08:28:57.839350+0000","last_clean":"2024-05-09T08:28:57.839350+0000","last_became_active":"2024-05-09T08:27:55.800700+0000","last_became_peered":"2024-05-09T08:27:55.800700+0000","last_unstale":"2024-05-09T08:28:57.839350+0000","last_undegraded":"2024-05-09T08:28:57.839350+0000","last_fullsized":"2024-05-09T08:28:57.839350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:44:42.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841255+0000","last_change":"2024-05-09T08:27:55.802266+0000","last_active":"2024-05-09T08:28:57.841255+0000","last_peered":"2024-05-09T08:28:57.841255+0000","last_clean":"2024-05-09T08:28:57.841255+0000","last_became_active":"2024-05-09T08:27:55.801670+0000","last_became_peered":"2024-05-09T08:27:55.801670+0000","last_unstale":"2024-05-09T08:28:57.841255+0000","last_undegraded":"2024-05-09T08:28:57.841255+0000","last_fullsized":"2024-05-09T08:28:57.841255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:53.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844724+0000","last_change":"2024-05-09T08:27:55.820107+0000","last_active":"2024-05-09T08:28:57.844724+0000","last_peered":"2024-05-09T08:28:57.844724+0000","last_clean":"2024-05-09T08:28:57.844724+0000","last_became_active":"2024-05-09T08:27:55.819748+0000","last_became_peered":"2024-05-09T08:27:55.819748+0000","last_unstale":"2024-05-09T08:28:57.844724+0000","last_undegraded":"2024-05-09T08:28:57.844724+0000","last_fullsized":"2024-05-09T08:28:57.844724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:08:04.311374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839635+0000","last_change":"2024-05-09T08:27:55.821625+0000","last_active":"2024-05-09T08:28:57.839635+0000","last_peered":"2024-05-09T08:28:57.839635+0000","last_clean":"2024-05-09T08:28:57.839635+0000","last_became_active":"2024-05-09T08:27:55.821291+0000","last_became_peered":"2024-05-09T08:27:55.821291+0000","last_unstale":"2024-05-09T08:28:57.839635+0000","last_undegraded":"2024-05-09T08:28:57.839635+0000","last_fullsized":"2024-05-09T08:28:57.839635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:40.017767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843250+0000","last_change":"2024-05-09T08:27:55.818212+0000","last_active":"2024-05-09T08:28:57.843250+0000","last_peered":"2024-05-09T08:28:57.843250+0000","last_clean":"2024-05-09T08:28:57.843250+0000","last_became_active":"2024-05-09T08:27:55.817874+0000","last_became_peered":"2024-05-09T08:27:55.817874+0000","last_unstale":"2024-05-09T08:28:57.843250+0000","last_undegraded":"2024-05-09T08:28:57.843250+0000","last_fullsized":"2024-05-09T08:28:57.843250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:20.189942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940739+0000","last_change":"2024-05-09T08:27:55.820128+0000","last_active":"2024-05-09T08:28:57.940739+0000","last_peered":"2024-05-09T08:28:57.940739+0000","last_clean":"2024-05-09T08:28:57.940739+0000","last_became_active":"2024-05-09T08:27:55.819717+0000","last_became_peered":"2024-05-09T08:27:55.819717+0000","last_unstale":"2024-05-09T08:28:57.940739+0000","last_undegraded":"2024-05-09T08:28:57.940739+0000","last_fullsized":"2024-05-09T08:28:57.940739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:48.324777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835270+0000","last_change":"2024-05-09T08:27:55.820639+0000","last_active":"2024-05-09T08:28:57.835270+0000","last_peered":"2024-05-09T08:28:57.835270+0000","last_clean":"2024-05-09T08:28:57.835270+0000","last_became_active":"2024-05-09T08:27:55.820116+0000","last_became_peered":"2024-05-09T08:27:55.820116+0000","last_unstale":"2024-05-09T08:28:57.835270+0000","last_undegraded":"2024-05-09T08:28:57.835270+0000","last_fullsized":"2024-05-09T08:28:57.835270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:35.099440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844163+0000","last_change":"2024-05-09T08:28:57.844163+0000","last_active":"2024-05-09T08:28:57.844163+0000","last_peered":"2024-05-09T08:28:57.844163+0000","last_clean":"2024-05-09T08:28:57.844163+0000","last_became_active":"2024-05-09T08:28:57.843753+0000","last_became_peered":"2024-05-09T08:28:57.843753+0000","last_unstale":"2024-05-09T08:28:57.844163+0000","last_undegraded":"2024-05-09T08:28:57.844163+0000","last_fullsized":"2024-05-09T08:28:57.844163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:31.238074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280679,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63466,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":501,"num_read_kb":318,"num_write":155,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":190398,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":459740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1111077,"data_compressed":37567,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":459896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1051826,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":518623,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":460636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":519973,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":459676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":460193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1051002,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60815,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61434,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:29:23.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-09T08:29:24.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:23 smithi160 ceph-mon[34070]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:24.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:23 smithi133 ceph-mon[29610]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:24.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:23 smithi188 ceph-mon[34645]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:24.277 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:24 smithi160 ceph-mon[34070]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:24 smithi133 ceph-mon[29610]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:24 smithi188 ceph-mon[34645]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:26.069 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:25 smithi133 ceph-mon[29610]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:26.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:25 smithi160 ceph-mon[34070]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:26.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:25 smithi188 ceph-mon[34645]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.593+0000 7f2580000700 1 -- 172.21.15.133:0/3265931937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 msgr2=0x7f2578115810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.593+0000 7f2580000700 1 --2- 172.21.15.133:0/3265931937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f2578115810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f25700098f0 tx=0x7f2570009c00 comp rx=0 tx=0).stop 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 -- 172.21.15.133:0/3265931937 shutdown_connections 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 --2- 172.21.15.133:0/3265931937 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f2578115810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 --2- 172.21.15.133:0/3265931937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f257810e940 0x7f257810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 --2- 172.21.15.133:0/3265931937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2578108990 0x7f2578108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 -- 172.21.15.133:0/3265931937 >> 172.21.15.133:0/3265931937 conn(0x7f2578075af0 msgr2=0x7f2578077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:26.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 -- 172.21.15.133:0/3265931937 shutdown_connections 2024-05-09T08:29:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.594+0000 7f2580000700 1 -- 172.21.15.133:0/3265931937 wait complete. 2024-05-09T08:29:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.595+0000 7f2580000700 1 Processor -- start 2024-05-09T08:29:26.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.595+0000 7f2580000700 1 -- start start 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 0x7f257819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f257810e940 0x7f25781a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2578118cc0 con 0x7f2578108990 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2578118b40 con 0x7f2578109350 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f2580000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2578118e40 con 0x7f257810e940 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257d600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 0x7f257819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257cc00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257cc00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:56856/0 (socket says 172.21.15.133:56856) 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257d600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 0x7f257819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45968/0 (socket says 172.21.15.133:45968) 2024-05-09T08:29:26.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257cc00700 1 -- 172.21.15.133:0/2651490996 learned_addr learned my addr 172.21.15.133:0/2651490996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.596+0000 7f257e000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f257810e940 0x7f25781a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 -- 172.21.15.133:0/2651490996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f257810e940 msgr2=0x7f25781a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:26.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f257810e940 0x7f25781a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 -- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 msgr2=0x7f257819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 0x7f257819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 -- 172.21.15.133:0/2651490996 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25781a4160 con 0x7f2578109350 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f257cc00700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9a0 tx=0x7f256400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256400ca10 con 0x7f2578109350 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256400cbb0 con 0x7f2578109350 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781a4450 con 0x7f2578109350 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564018680 con 0x7f2578109350 2024-05-09T08:29:26.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.597+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781abcd0 con 0x7f2578109350 2024-05-09T08:29:26.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.599+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2578069010 con 0x7f2578109350 2024-05-09T08:29:26.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.601+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2564010430 con 0x7f2578109350 2024-05-09T08:29:26.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.602+0000 7f2576000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 0x7f25600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:26.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.603+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f25640cf8f0 con 0x7f2578109350 2024-05-09T08:29:26.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.603+0000 7f257d600700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 0x7f25600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:26.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.603+0000 7f257d600700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 0x7f25600a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f256c009e60 tx=0x7f256c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:26.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.604+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f256409c6c0 con 0x7f2578109350 2024-05-09T08:29:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.858+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2578062760 con 0x7f25600a2cb0 2024-05-09T08:29:26.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.867+0000 7f2576000700 1 -- 172.21.15.133:0/2651490996 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258933 (secure 0 0 0) 0x7f2578062760 con 0x7f25600a2cb0 2024-05-09T08:29:26.873 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 msgr2=0x7f25600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 0x7f25600a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f256c009e60 tx=0x7f256c00b040 comp rx=0 tx=0).stop 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 msgr2=0x7f257819f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9a0 tx=0x7f256400ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 shutdown_connections 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25600a2cb0 0x7f25600a5170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f257810e940 0x7f25781a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2578108990 0x7f257819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 --2- 172.21.15.133:0/2651490996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2578109350 0x7f257819f690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 >> 172.21.15.133:0/2651490996 conn(0x7f2578075af0 msgr2=0x7f25780760d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 shutdown_connections 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:26.873+0000 7f2580000700 1 -- 172.21.15.133:0/2651490996 wait complete. 2024-05-09T08:29:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-05-09T08:29:27.732 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-05-09T08:29:26.696940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654068,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":657,"num_read_kb":442,"num_write":338,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373364,"kb_used_data":11652,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568460,"statfs":{"total":1151940427776,"available":1151558103040,"internally_reserved":0,"allocated":11931648,"data_stored":7472100,"data_compressed":112701,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005163"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839179+0000","last_change":"2024-05-09T08:27:55.828188+0000","last_active":"2024-05-09T08:28:57.839179+0000","last_peered":"2024-05-09T08:28:57.839179+0000","last_clean":"2024-05-09T08:28:57.839179+0000","last_became_active":"2024-05-09T08:27:55.827858+0000","last_became_peered":"2024-05-09T08:27:55.827858+0000","last_unstale":"2024-05-09T08:28:57.839179+0000","last_undegraded":"2024-05-09T08:28:57.839179+0000","last_fullsized":"2024-05-09T08:28:57.839179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:54:19.880601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833309+0000","last_change":"2024-05-09T08:27:55.821504+0000","last_active":"2024-05-09T08:28:57.833309+0000","last_peered":"2024-05-09T08:28:57.833309+0000","last_clean":"2024-05-09T08:28:57.833309+0000","last_became_active":"2024-05-09T08:27:55.818102+0000","last_became_peered":"2024-05-09T08:27:55.818102+0000","last_unstale":"2024-05-09T08:28:57.833309+0000","last_undegraded":"2024-05-09T08:28:57.833309+0000","last_fullsized":"2024-05-09T08:28:57.833309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:15:00.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833859+0000","last_change":"2024-05-09T08:27:55.817240+0000","last_active":"2024-05-09T08:28:57.833859+0000","last_peered":"2024-05-09T08:28:57.833859+0000","last_clean":"2024-05-09T08:28:57.833859+0000","last_became_active":"2024-05-09T08:27:55.816865+0000","last_became_peered":"2024-05-09T08:27:55.816865+0000","last_unstale":"2024-05-09T08:28:57.833859+0000","last_undegraded":"2024-05-09T08:28:57.833859+0000","last_fullsized":"2024-05-09T08:28:57.833859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:04.149531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940132+0000","last_change":"2024-05-09T08:27:55.805704+0000","last_active":"2024-05-09T08:28:57.940132+0000","last_peered":"2024-05-09T08:28:57.940132+0000","last_clean":"2024-05-09T08:28:57.940132+0000","last_became_active":"2024-05-09T08:27:55.805344+0000","last_became_peered":"2024-05-09T08:27:55.805344+0000","last_unstale":"2024-05-09T08:28:57.940132+0000","last_undegraded":"2024-05-09T08:28:57.940132+0000","last_fullsized":"2024-05-09T08:28:57.940132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:02.418060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842021+0000","last_change":"2024-05-09T08:27:55.821747+0000","last_active":"2024-05-09T08:28:57.842021+0000","last_peered":"2024-05-09T08:28:57.842021+0000","last_clean":"2024-05-09T08:28:57.842021+0000","last_became_active":"2024-05-09T08:27:55.821378+0000","last_became_peered":"2024-05-09T08:27:55.821378+0000","last_unstale":"2024-05-09T08:28:57.842021+0000","last_undegraded":"2024-05-09T08:28:57.842021+0000","last_fullsized":"2024-05-09T08:28:57.842021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:53.470723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825644+0000","last_change":"2024-05-09T08:27:55.812827+0000","last_active":"2024-05-09T08:28:57.825644+0000","last_peered":"2024-05-09T08:28:57.825644+0000","last_clean":"2024-05-09T08:28:57.825644+0000","last_became_active":"2024-05-09T08:27:55.812412+0000","last_became_peered":"2024-05-09T08:27:55.812412+0000","last_unstale":"2024-05-09T08:28:57.825644+0000","last_undegraded":"2024-05-09T08:28:57.825644+0000","last_fullsized":"2024-05-09T08:28:57.825644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:20.178719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838677+0000","last_change":"2024-05-09T08:27:55.811536+0000","last_active":"2024-05-09T08:28:57.838677+0000","last_peered":"2024-05-09T08:28:57.838677+0000","last_clean":"2024-05-09T08:28:57.838677+0000","last_became_active":"2024-05-09T08:27:55.811181+0000","last_became_peered":"2024-05-09T08:27:55.811181+0000","last_unstale":"2024-05-09T08:28:57.838677+0000","last_undegraded":"2024-05-09T08:28:57.838677+0000","last_fullsized":"2024-05-09T08:28:57.838677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:38:48.478456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.845386+0000","last_change":"2024-05-09T08:28:57.842336+0000","last_active":"2024-05-09T08:28:57.845386+0000","last_peered":"2024-05-09T08:28:57.845386+0000","last_clean":"2024-05-09T08:28:57.845386+0000","last_became_active":"2024-05-09T08:28:57.841559+0000","last_became_peered":"2024-05-09T08:28:57.841559+0000","last_unstale":"2024-05-09T08:28:57.845386+0000","last_undegraded":"2024-05-09T08:28:57.845386+0000","last_fullsized":"2024-05-09T08:28:57.845386+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:34.208823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.941205+0000","last_change":"2024-05-09T08:27:55.802154+0000","last_active":"2024-05-09T08:28:57.941205+0000","last_peered":"2024-05-09T08:28:57.941205+0000","last_clean":"2024-05-09T08:28:57.941205+0000","last_became_active":"2024-05-09T08:27:55.801501+0000","last_became_peered":"2024-05-09T08:27:55.801501+0000","last_unstale":"2024-05-09T08:28:57.941205+0000","last_undegraded":"2024-05-09T08:28:57.941205+0000","last_fullsized":"2024-05-09T08:28:57.941205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:42.009521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841929+0000","last_change":"2024-05-09T08:28:57.841929+0000","last_active":"2024-05-09T08:28:57.841929+0000","last_peered":"2024-05-09T08:28:57.841929+0000","last_clean":"2024-05-09T08:28:57.841929+0000","last_became_active":"2024-05-09T08:28:57.841717+0000","last_became_peered":"2024-05-09T08:28:57.841717+0000","last_unstale":"2024-05-09T08:28:57.841929+0000","last_undegraded":"2024-05-09T08:28:57.841929+0000","last_fullsized":"2024-05-09T08:28:57.841929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:42.099158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838874+0000","last_change":"2024-05-09T08:27:55.806122+0000","last_active":"2024-05-09T08:28:57.838874+0000","last_peered":"2024-05-09T08:28:57.838874+0000","last_clean":"2024-05-09T08:28:57.838874+0000","last_became_active":"2024-05-09T08:27:55.803844+0000","last_became_peered":"2024-05-09T08:27:55.803844+0000","last_unstale":"2024-05-09T08:28:57.838874+0000","last_undegraded":"2024-05-09T08:28:57.838874+0000","last_fullsized":"2024-05-09T08:28:57.838874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:40.692633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940023+0000","last_change":"2024-05-09T08:27:55.810184+0000","last_active":"2024-05-09T08:28:57.940023+0000","last_peered":"2024-05-09T08:28:57.940023+0000","last_clean":"2024-05-09T08:28:57.940023+0000","last_became_active":"2024-05-09T08:27:55.809833+0000","last_became_peered":"2024-05-09T08:27:55.809833+0000","last_unstale":"2024-05-09T08:28:57.940023+0000","last_undegraded":"2024-05-09T08:28:57.940023+0000","last_fullsized":"2024-05-09T08:28:57.940023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:19:33.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835264+0000","last_change":"2024-05-09T08:27:55.804770+0000","last_active":"2024-05-09T08:28:57.835264+0000","last_peered":"2024-05-09T08:28:57.835264+0000","last_clean":"2024-05-09T08:28:57.835264+0000","last_became_active":"2024-05-09T08:27:55.803486+0000","last_became_peered":"2024-05-09T08:27:55.803486+0000","last_unstale":"2024-05-09T08:28:57.835264+0000","last_undegraded":"2024-05-09T08:28:57.835264+0000","last_fullsized":"2024-05-09T08:28:57.835264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:28.641874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.832338+0000","last_change":"2024-05-09T08:27:55.826032+0000","last_active":"2024-05-09T08:28:57.832338+0000","last_peered":"2024-05-09T08:28:57.832338+0000","last_clean":"2024-05-09T08:28:57.832338+0000","last_became_active":"2024-05-09T08:27:55.825510+0000","last_became_peered":"2024-05-09T08:27:55.825510+0000","last_unstale":"2024-05-09T08:28:57.832338+0000","last_undegraded":"2024-05-09T08:28:57.832338+0000","last_fullsized":"2024-05-09T08:28:57.832338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:25.691361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.831679+0000","last_change":"2024-05-09T08:27:55.816178+0000","last_active":"2024-05-09T08:28:57.831679+0000","last_peered":"2024-05-09T08:28:57.831679+0000","last_clean":"2024-05-09T08:28:57.831679+0000","last_became_active":"2024-05-09T08:27:55.814592+0000","last_became_peered":"2024-05-09T08:27:55.814592+0000","last_unstale":"2024-05-09T08:28:57.831679+0000","last_undegraded":"2024-05-09T08:28:57.831679+0000","last_fullsized":"2024-05-09T08:28:57.831679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:06.632314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940772+0000","last_change":"2024-05-09T08:27:55.827914+0000","last_active":"2024-05-09T08:28:57.940772+0000","last_peered":"2024-05-09T08:28:57.940772+0000","last_clean":"2024-05-09T08:28:57.940772+0000","last_became_active":"2024-05-09T08:27:55.827555+0000","last_became_peered":"2024-05-09T08:27:55.827555+0000","last_unstale":"2024-05-09T08:28:57.940772+0000","last_undegraded":"2024-05-09T08:28:57.940772+0000","last_fullsized":"2024-05-09T08:28:57.940772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:37.097461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839443+0000","last_change":"2024-05-09T08:27:55.828072+0000","last_active":"2024-05-09T08:28:57.839443+0000","last_peered":"2024-05-09T08:28:57.839443+0000","last_clean":"2024-05-09T08:28:57.839443+0000","last_became_active":"2024-05-09T08:27:55.827727+0000","last_became_peered":"2024-05-09T08:27:55.827727+0000","last_unstale":"2024-05-09T08:28:57.839443+0000","last_undegraded":"2024-05-09T08:28:57.839443+0000","last_fullsized":"2024-05-09T08:28:57.839443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:29:13.761560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.829550+0000","last_change":"2024-05-09T08:27:55.812506+0000","last_active":"2024-05-09T08:28:57.829550+0000","last_peered":"2024-05-09T08:28:57.829550+0000","last_clean":"2024-05-09T08:28:57.829550+0000","last_became_active":"2024-05-09T08:27:55.812071+0000","last_became_peered":"2024-05-09T08:27:55.812071+0000","last_unstale":"2024-05-09T08:28:57.829550+0000","last_undegraded":"2024-05-09T08:28:57.829550+0000","last_fullsized":"2024-05-09T08:28:57.829550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:40:53.318140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841325+0000","last_change":"2024-05-09T08:27:55.817660+0000","last_active":"2024-05-09T08:28:57.841325+0000","last_peered":"2024-05-09T08:28:57.841325+0000","last_clean":"2024-05-09T08:28:57.841325+0000","last_became_active":"2024-05-09T08:27:55.817357+0000","last_became_peered":"2024-05-09T08:27:55.817357+0000","last_unstale":"2024-05-09T08:28:57.841325+0000","last_undegraded":"2024-05-09T08:28:57.841325+0000","last_fullsized":"2024-05-09T08:28:57.841325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:05.489063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.830096+0000","last_change":"2024-05-09T08:27:54.784177+0000","last_active":"2024-05-09T08:28:57.830096+0000","last_peered":"2024-05-09T08:28:57.830096+0000","last_clean":"2024-05-09T08:28:57.830096+0000","last_became_active":"2024-05-09T08:27:54.780982+0000","last_became_peered":"2024-05-09T08:27:54.780982+0000","last_unstale":"2024-05-09T08:28:57.830096+0000","last_undegraded":"2024-05-09T08:28:57.830096+0000","last_fullsized":"2024-05-09T08:28:57.830096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:17.668694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844135+0000","last_change":"2024-05-09T08:27:55.825495+0000","last_active":"2024-05-09T08:28:57.844135+0000","last_peered":"2024-05-09T08:28:57.844135+0000","last_clean":"2024-05-09T08:28:57.844135+0000","last_became_active":"2024-05-09T08:27:55.821680+0000","last_became_peered":"2024-05-09T08:27:55.821680+0000","last_unstale":"2024-05-09T08:28:57.844135+0000","last_undegraded":"2024-05-09T08:28:57.844135+0000","last_fullsized":"2024-05-09T08:28:57.844135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:46:06.128200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844264+0000","last_change":"2024-05-09T08:27:54.791362+0000","last_active":"2024-05-09T08:28:57.844264+0000","last_peered":"2024-05-09T08:28:57.844264+0000","last_clean":"2024-05-09T08:28:57.844264+0000","last_became_active":"2024-05-09T08:27:54.791050+0000","last_became_peered":"2024-05-09T08:27:54.791050+0000","last_unstale":"2024-05-09T08:28:57.844264+0000","last_undegraded":"2024-05-09T08:28:57.844264+0000","last_fullsized":"2024-05-09T08:28:57.844264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:45:11.821907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835743+0000","last_change":"2024-05-09T08:27:55.820278+0000","last_active":"2024-05-09T08:28:57.835743+0000","last_peered":"2024-05-09T08:28:57.835743+0000","last_clean":"2024-05-09T08:28:57.835743+0000","last_became_active":"2024-05-09T08:27:55.819861+0000","last_became_peered":"2024-05-09T08:27:55.819861+0000","last_unstale":"2024-05-09T08:28:57.835743+0000","last_undegraded":"2024-05-09T08:28:57.835743+0000","last_fullsized":"2024-05-09T08:28:57.835743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:29.541259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843371+0000","last_change":"2024-05-09T08:27:54.771453+0000","last_active":"2024-05-09T08:28:57.843371+0000","last_peered":"2024-05-09T08:28:57.843371+0000","last_clean":"2024-05-09T08:28:57.843371+0000","last_became_active":"2024-05-09T08:27:54.770746+0000","last_became_peered":"2024-05-09T08:27:54.770746+0000","last_unstale":"2024-05-09T08:28:57.843371+0000","last_undegraded":"2024-05-09T08:28:57.843371+0000","last_fullsized":"2024-05-09T08:28:57.843371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:20:18.823562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940718+0000","last_change":"2024-05-09T08:27:55.803341+0000","last_active":"2024-05-09T08:28:57.940718+0000","last_peered":"2024-05-09T08:28:57.940718+0000","last_clean":"2024-05-09T08:28:57.940718+0000","last_became_active":"2024-05-09T08:27:55.802945+0000","last_became_peered":"2024-05-09T08:27:55.802945+0000","last_unstale":"2024-05-09T08:28:57.940718+0000","last_undegraded":"2024-05-09T08:28:57.940718+0000","last_fullsized":"2024-05-09T08:28:57.940718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:24.346139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843074+0000","last_change":"2024-05-09T08:27:54.769140+0000","last_active":"2024-05-09T08:28:57.843074+0000","last_peered":"2024-05-09T08:28:57.843074+0000","last_clean":"2024-05-09T08:28:57.843074+0000","last_became_active":"2024-05-09T08:27:54.768615+0000","last_became_peered":"2024-05-09T08:27:54.768615+0000","last_unstale":"2024-05-09T08:28:57.843074+0000","last_undegraded":"2024-05-09T08:28:57.843074+0000","last_fullsized":"2024-05-09T08:28:57.843074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:10.016070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838835+0000","last_change":"2024-05-09T08:27:55.806385+0000","last_active":"2024-05-09T08:28:57.838835+0000","last_peered":"2024-05-09T08:28:57.838835+0000","last_clean":"2024-05-09T08:28:57.838835+0000","last_became_active":"2024-05-09T08:27:55.806031+0000","last_became_peered":"2024-05-09T08:27:55.806031+0000","last_unstale":"2024-05-09T08:28:57.838835+0000","last_undegraded":"2024-05-09T08:28:57.838835+0000","last_fullsized":"2024-05-09T08:28:57.838835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:00:17.326277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:22.402333+0000","last_change":"2024-05-09T08:27:54.769148+0000","last_active":"2024-05-09T08:29:22.402333+0000","last_peered":"2024-05-09T08:29:22.402333+0000","last_clean":"2024-05-09T08:29:22.402333+0000","last_became_active":"2024-05-09T08:27:54.768814+0000","last_became_peered":"2024-05-09T08:27:54.768814+0000","last_unstale":"2024-05-09T08:29:22.402333+0000","last_undegraded":"2024-05-09T08:29:22.402333+0000","last_fullsized":"2024-05-09T08:29:22.402333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:08.276294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842085+0000","last_change":"2024-05-09T08:27:55.812884+0000","last_active":"2024-05-09T08:28:57.842085+0000","last_peered":"2024-05-09T08:28:57.842085+0000","last_clean":"2024-05-09T08:28:57.842085+0000","last_became_active":"2024-05-09T08:27:55.812515+0000","last_became_peered":"2024-05-09T08:27:55.812515+0000","last_unstale":"2024-05-09T08:28:57.842085+0000","last_undegraded":"2024-05-09T08:28:57.842085+0000","last_fullsized":"2024-05-09T08:28:57.842085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:59.211293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825569+0000","last_change":"2024-05-09T08:27:54.764806+0000","last_active":"2024-05-09T08:28:57.825569+0000","last_peered":"2024-05-09T08:28:57.825569+0000","last_clean":"2024-05-09T08:28:57.825569+0000","last_became_active":"2024-05-09T08:27:54.764325+0000","last_became_peered":"2024-05-09T08:27:54.764325+0000","last_unstale":"2024-05-09T08:28:57.825569+0000","last_undegraded":"2024-05-09T08:28:57.825569+0000","last_fullsized":"2024-05-09T08:28:57.825569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:11.820240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838697+0000","last_change":"2024-05-09T08:27:55.822481+0000","last_active":"2024-05-09T08:28:57.838697+0000","last_peered":"2024-05-09T08:28:57.838697+0000","last_clean":"2024-05-09T08:28:57.838697+0000","last_became_active":"2024-05-09T08:27:55.821180+0000","last_became_peered":"2024-05-09T08:27:55.821180+0000","last_unstale":"2024-05-09T08:28:57.838697+0000","last_undegraded":"2024-05-09T08:28:57.838697+0000","last_fullsized":"2024-05-09T08:28:57.838697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:33:46.382705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834771+0000","last_change":"2024-05-09T08:27:54.771263+0000","last_active":"2024-05-09T08:28:57.834771+0000","last_peered":"2024-05-09T08:28:57.834771+0000","last_clean":"2024-05-09T08:28:57.834771+0000","last_became_active":"2024-05-09T08:27:54.770719+0000","last_became_peered":"2024-05-09T08:27:54.770719+0000","last_unstale":"2024-05-09T08:28:57.834771+0000","last_undegraded":"2024-05-09T08:28:57.834771+0000","last_fullsized":"2024-05-09T08:28:57.834771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:08:08.725081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842207+0000","last_change":"2024-05-09T08:27:55.821493+0000","last_active":"2024-05-09T08:28:57.842207+0000","last_peered":"2024-05-09T08:28:57.842207+0000","last_clean":"2024-05-09T08:28:57.842207+0000","last_became_active":"2024-05-09T08:27:55.818585+0000","last_became_peered":"2024-05-09T08:27:55.818585+0000","last_unstale":"2024-05-09T08:28:57.842207+0000","last_undegraded":"2024-05-09T08:28:57.842207+0000","last_fullsized":"2024-05-09T08:28:57.842207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:37:56.319691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839532+0000","last_change":"2024-05-09T08:27:54.769856+0000","last_active":"2024-05-09T08:28:57.839532+0000","last_peered":"2024-05-09T08:28:57.839532+0000","last_clean":"2024-05-09T08:28:57.839532+0000","last_became_active":"2024-05-09T08:27:54.769548+0000","last_became_peered":"2024-05-09T08:27:54.769548+0000","last_unstale":"2024-05-09T08:28:57.839532+0000","last_undegraded":"2024-05-09T08:28:57.839532+0000","last_fullsized":"2024-05-09T08:28:57.839532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:57.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.828624+0000","last_change":"2024-05-09T08:27:55.822063+0000","last_active":"2024-05-09T08:28:57.828624+0000","last_peered":"2024-05-09T08:28:57.828624+0000","last_clean":"2024-05-09T08:28:57.828624+0000","last_became_active":"2024-05-09T08:27:55.821624+0000","last_became_peered":"2024-05-09T08:27:55.821624+0000","last_unstale":"2024-05-09T08:28:57.828624+0000","last_undegraded":"2024-05-09T08:28:57.828624+0000","last_fullsized":"2024-05-09T08:28:57.828624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:14.876870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:25.406307+0000","last_change":"2024-05-09T08:27:54.767639+0000","last_active":"2024-05-09T08:29:25.406307+0000","last_peered":"2024-05-09T08:29:25.406307+0000","last_clean":"2024-05-09T08:29:25.406307+0000","last_became_active":"2024-05-09T08:27:54.767284+0000","last_became_peered":"2024-05-09T08:27:54.767284+0000","last_unstale":"2024-05-09T08:29:25.406307+0000","last_undegraded":"2024-05-09T08:29:25.406307+0000","last_fullsized":"2024-05-09T08:29:25.406307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:07:07.110319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940992+0000","last_change":"2024-05-09T08:27:55.817050+0000","last_active":"2024-05-09T08:28:57.940992+0000","last_peered":"2024-05-09T08:28:57.940992+0000","last_clean":"2024-05-09T08:28:57.940992+0000","last_became_active":"2024-05-09T08:27:55.816655+0000","last_became_peered":"2024-05-09T08:27:55.816655+0000","last_unstale":"2024-05-09T08:28:57.940992+0000","last_undegraded":"2024-05-09T08:28:57.940992+0000","last_fullsized":"2024-05-09T08:28:57.940992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:48:06.132482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838870+0000","last_change":"2024-05-09T08:27:54.770449+0000","last_active":"2024-05-09T08:28:57.838870+0000","last_peered":"2024-05-09T08:28:57.838870+0000","last_clean":"2024-05-09T08:28:57.838870+0000","last_became_active":"2024-05-09T08:27:54.769382+0000","last_became_peered":"2024-05-09T08:27:54.769382+0000","last_unstale":"2024-05-09T08:28:57.838870+0000","last_undegraded":"2024-05-09T08:28:57.838870+0000","last_fullsized":"2024-05-09T08:28:57.838870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:24:47.108749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834387+0000","last_change":"2024-05-09T08:27:55.831416+0000","last_active":"2024-05-09T08:28:57.834387+0000","last_peered":"2024-05-09T08:28:57.834387+0000","last_clean":"2024-05-09T08:28:57.834387+0000","last_became_active":"2024-05-09T08:27:55.831011+0000","last_became_peered":"2024-05-09T08:27:55.831011+0000","last_unstale":"2024-05-09T08:28:57.834387+0000","last_undegraded":"2024-05-09T08:28:57.834387+0000","last_fullsized":"2024-05-09T08:28:57.834387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:23.820425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838929+0000","last_change":"2024-05-09T08:27:54.776737+0000","last_active":"2024-05-09T08:28:57.838929+0000","last_peered":"2024-05-09T08:28:57.838929+0000","last_clean":"2024-05-09T08:28:57.838929+0000","last_became_active":"2024-05-09T08:27:54.776402+0000","last_became_peered":"2024-05-09T08:27:54.776402+0000","last_unstale":"2024-05-09T08:28:57.838929+0000","last_undegraded":"2024-05-09T08:28:57.838929+0000","last_fullsized":"2024-05-09T08:28:57.838929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:09:38.135159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940931+0000","last_change":"2024-05-09T08:27:55.810092+0000","last_active":"2024-05-09T08:28:57.940931+0000","last_peered":"2024-05-09T08:28:57.940931+0000","last_clean":"2024-05-09T08:28:57.940931+0000","last_became_active":"2024-05-09T08:27:55.809738+0000","last_became_peered":"2024-05-09T08:27:55.809738+0000","last_unstale":"2024-05-09T08:28:57.940931+0000","last_undegraded":"2024-05-09T08:28:57.940931+0000","last_fullsized":"2024-05-09T08:28:57.940931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:34.094014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":225,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:26.413047+0000","last_change":"2024-05-09T08:27:54.769608+0000","last_active":"2024-05-09T08:29:26.413047+0000","last_peered":"2024-05-09T08:29:26.413047+0000","last_clean":"2024-05-09T08:29:26.413047+0000","last_became_active":"2024-05-09T08:27:54.769286+0000","last_became_peered":"2024-05-09T08:27:54.769286+0000","last_unstale":"2024-05-09T08:29:26.413047+0000","last_undegraded":"2024-05-09T08:29:26.413047+0000","last_fullsized":"2024-05-09T08:29:26.413047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:20:40.620602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":179,"num_read_kb":147,"num_write":28,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842882+0000","last_change":"2024-05-09T08:27:55.800489+0000","last_active":"2024-05-09T08:28:57.842882+0000","last_peered":"2024-05-09T08:28:57.842882+0000","last_clean":"2024-05-09T08:28:57.842882+0000","last_became_active":"2024-05-09T08:27:55.800145+0000","last_became_peered":"2024-05-09T08:27:55.800145+0000","last_unstale":"2024-05-09T08:28:57.842882+0000","last_undegraded":"2024-05-09T08:28:57.842882+0000","last_fullsized":"2024-05-09T08:28:57.842882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:54:47.877630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841014+0000","last_change":"2024-05-09T08:27:54.779179+0000","last_active":"2024-05-09T08:28:57.841014+0000","last_peered":"2024-05-09T08:28:57.841014+0000","last_clean":"2024-05-09T08:28:57.841014+0000","last_became_active":"2024-05-09T08:27:54.778793+0000","last_became_peered":"2024-05-09T08:27:54.778793+0000","last_unstale":"2024-05-09T08:28:57.841014+0000","last_undegraded":"2024-05-09T08:28:57.841014+0000","last_fullsized":"2024-05-09T08:28:57.841014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:56:42.414959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835498+0000","last_change":"2024-05-09T08:27:55.830500+0000","last_active":"2024-05-09T08:28:57.835498+0000","last_peered":"2024-05-09T08:28:57.835498+0000","last_clean":"2024-05-09T08:28:57.835498+0000","last_became_active":"2024-05-09T08:27:55.830044+0000","last_became_peered":"2024-05-09T08:27:55.830044+0000","last_unstale":"2024-05-09T08:28:57.835498+0000","last_undegraded":"2024-05-09T08:28:57.835498+0000","last_fullsized":"2024-05-09T08:28:57.835498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:15.616303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838889+0000","last_change":"2024-05-09T08:27:54.786607+0000","last_active":"2024-05-09T08:28:57.838889+0000","last_peered":"2024-05-09T08:28:57.838889+0000","last_clean":"2024-05-09T08:28:57.838889+0000","last_became_active":"2024-05-09T08:27:54.786286+0000","last_became_peered":"2024-05-09T08:27:54.786286+0000","last_unstale":"2024-05-09T08:28:57.838889+0000","last_undegraded":"2024-05-09T08:28:57.838889+0000","last_fullsized":"2024-05-09T08:28:57.838889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:17.833237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838921+0000","last_change":"2024-05-09T08:27:55.809022+0000","last_active":"2024-05-09T08:28:57.838921+0000","last_peered":"2024-05-09T08:28:57.838921+0000","last_clean":"2024-05-09T08:28:57.838921+0000","last_became_active":"2024-05-09T08:27:55.808666+0000","last_became_peered":"2024-05-09T08:27:55.808666+0000","last_unstale":"2024-05-09T08:28:57.838921+0000","last_undegraded":"2024-05-09T08:28:57.838921+0000","last_fullsized":"2024-05-09T08:28:57.838921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:18.339576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:22.456782+0000","last_change":"2024-05-09T08:27:54.769166+0000","last_active":"2024-05-09T08:29:22.456782+0000","last_peered":"2024-05-09T08:29:22.456782+0000","last_clean":"2024-05-09T08:29:22.456782+0000","last_became_active":"2024-05-09T08:27:54.768794+0000","last_became_peered":"2024-05-09T08:27:54.768794+0000","last_unstale":"2024-05-09T08:29:22.456782+0000","last_undegraded":"2024-05-09T08:29:22.456782+0000","last_fullsized":"2024-05-09T08:29:22.456782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:25:25.222388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838602+0000","last_change":"2024-05-09T08:27:55.806685+0000","last_active":"2024-05-09T08:28:57.838602+0000","last_peered":"2024-05-09T08:28:57.838602+0000","last_clean":"2024-05-09T08:28:57.838602+0000","last_became_active":"2024-05-09T08:27:55.804641+0000","last_became_peered":"2024-05-09T08:27:55.804641+0000","last_unstale":"2024-05-09T08:28:57.838602+0000","last_undegraded":"2024-05-09T08:28:57.838602+0000","last_fullsized":"2024-05-09T08:28:57.838602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:28:40.307764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842320+0000","last_change":"2024-05-09T08:27:54.782524+0000","last_active":"2024-05-09T08:28:57.842320+0000","last_peered":"2024-05-09T08:28:57.842320+0000","last_clean":"2024-05-09T08:28:57.842320+0000","last_became_active":"2024-05-09T08:27:54.782220+0000","last_became_peered":"2024-05-09T08:27:54.782220+0000","last_unstale":"2024-05-09T08:28:57.842320+0000","last_undegraded":"2024-05-09T08:28:57.842320+0000","last_fullsized":"2024-05-09T08:28:57.842320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:16:58.705682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838632+0000","last_change":"2024-05-09T08:27:55.806390+0000","last_active":"2024-05-09T08:28:57.838632+0000","last_peered":"2024-05-09T08:28:57.838632+0000","last_clean":"2024-05-09T08:28:57.838632+0000","last_became_active":"2024-05-09T08:27:55.806027+0000","last_became_peered":"2024-05-09T08:27:55.806027+0000","last_unstale":"2024-05-09T08:28:57.838632+0000","last_undegraded":"2024-05-09T08:28:57.838632+0000","last_fullsized":"2024-05-09T08:28:57.838632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:58.251839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842884+0000","last_change":"2024-05-09T08:27:54.766679+0000","last_active":"2024-05-09T08:28:57.842884+0000","last_peered":"2024-05-09T08:28:57.842884+0000","last_clean":"2024-05-09T08:28:57.842884+0000","last_became_active":"2024-05-09T08:27:54.766318+0000","last_became_peered":"2024-05-09T08:27:54.766318+0000","last_unstale":"2024-05-09T08:28:57.842884+0000","last_undegraded":"2024-05-09T08:28:57.842884+0000","last_fullsized":"2024-05-09T08:28:57.842884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:08:38.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838541+0000","last_change":"2024-05-09T08:27:55.804379+0000","last_active":"2024-05-09T08:28:57.838541+0000","last_peered":"2024-05-09T08:28:57.838541+0000","last_clean":"2024-05-09T08:28:57.838541+0000","last_became_active":"2024-05-09T08:27:55.804046+0000","last_became_peered":"2024-05-09T08:27:55.804046+0000","last_unstale":"2024-05-09T08:28:57.838541+0000","last_undegraded":"2024-05-09T08:28:57.838541+0000","last_fullsized":"2024-05-09T08:28:57.838541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:43.735487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.831056+0000","last_change":"2024-05-09T08:27:54.785336+0000","last_active":"2024-05-09T08:28:57.831056+0000","last_peered":"2024-05-09T08:28:57.831056+0000","last_clean":"2024-05-09T08:28:57.831056+0000","last_became_active":"2024-05-09T08:27:54.784581+0000","last_became_peered":"2024-05-09T08:27:54.784581+0000","last_unstale":"2024-05-09T08:28:57.831056+0000","last_undegraded":"2024-05-09T08:28:57.831056+0000","last_fullsized":"2024-05-09T08:28:57.831056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:40.071747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825920+0000","last_change":"2024-05-09T08:27:55.821512+0000","last_active":"2024-05-09T08:28:57.825920+0000","last_peered":"2024-05-09T08:28:57.825920+0000","last_clean":"2024-05-09T08:28:57.825920+0000","last_became_active":"2024-05-09T08:27:55.821162+0000","last_became_peered":"2024-05-09T08:27:55.821162+0000","last_unstale":"2024-05-09T08:28:57.825920+0000","last_undegraded":"2024-05-09T08:28:57.825920+0000","last_fullsized":"2024-05-09T08:28:57.825920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:47.377452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838776+0000","last_change":"2024-05-09T08:27:54.771528+0000","last_active":"2024-05-09T08:28:57.838776+0000","last_peered":"2024-05-09T08:28:57.838776+0000","last_clean":"2024-05-09T08:28:57.838776+0000","last_became_active":"2024-05-09T08:27:54.771133+0000","last_became_peered":"2024-05-09T08:27:54.771133+0000","last_unstale":"2024-05-09T08:28:57.838776+0000","last_undegraded":"2024-05-09T08:28:57.838776+0000","last_fullsized":"2024-05-09T08:28:57.838776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:58:22.948800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.825795+0000","last_change":"2024-05-09T08:27:55.824954+0000","last_active":"2024-05-09T08:28:57.825795+0000","last_peered":"2024-05-09T08:28:57.825795+0000","last_clean":"2024-05-09T08:28:57.825795+0000","last_became_active":"2024-05-09T08:27:55.824578+0000","last_became_peered":"2024-05-09T08:27:55.824578+0000","last_unstale":"2024-05-09T08:28:57.825795+0000","last_undegraded":"2024-05-09T08:28:57.825795+0000","last_fullsized":"2024-05-09T08:28:57.825795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:35:22.790193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844413+0000","last_change":"2024-05-09T08:28:57.844413+0000","last_active":"2024-05-09T08:28:57.844413+0000","last_peered":"2024-05-09T08:28:57.844413+0000","last_clean":"2024-05-09T08:28:57.844413+0000","last_became_active":"2024-05-09T08:28:57.844057+0000","last_became_peered":"2024-05-09T08:28:57.844057+0000","last_unstale":"2024-05-09T08:28:57.844413+0000","last_undegraded":"2024-05-09T08:28:57.844413+0000","last_fullsized":"2024-05-09T08:28:57.844413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:55.603293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:29:22.455983+0000","last_change":"2024-05-09T08:27:54.780486+0000","last_active":"2024-05-09T08:29:22.455983+0000","last_peered":"2024-05-09T08:29:22.455983+0000","last_clean":"2024-05-09T08:29:22.455983+0000","last_became_active":"2024-05-09T08:27:54.777349+0000","last_became_peered":"2024-05-09T08:27:54.777349+0000","last_unstale":"2024-05-09T08:29:22.455983+0000","last_undegraded":"2024-05-09T08:29:22.455983+0000","last_fullsized":"2024-05-09T08:29:22.455983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:04:14.729741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844568+0000","last_change":"2024-05-09T08:27:55.825469+0000","last_active":"2024-05-09T08:28:57.844568+0000","last_peered":"2024-05-09T08:28:57.844568+0000","last_clean":"2024-05-09T08:28:57.844568+0000","last_became_active":"2024-05-09T08:27:55.825103+0000","last_became_peered":"2024-05-09T08:27:55.825103+0000","last_unstale":"2024-05-09T08:28:57.844568+0000","last_undegraded":"2024-05-09T08:28:57.844568+0000","last_fullsized":"2024-05-09T08:28:57.844568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:53.136741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.939761+0000","last_change":"2024-05-09T08:27:54.774194+0000","last_active":"2024-05-09T08:28:57.939761+0000","last_peered":"2024-05-09T08:28:57.939761+0000","last_clean":"2024-05-09T08:28:57.939761+0000","last_became_active":"2024-05-09T08:27:54.773678+0000","last_became_peered":"2024-05-09T08:27:54.773678+0000","last_unstale":"2024-05-09T08:28:57.939761+0000","last_undegraded":"2024-05-09T08:28:57.939761+0000","last_fullsized":"2024-05-09T08:28:57.939761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:23.199301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838704+0000","last_change":"2024-05-09T08:26:34.966221+0000","last_active":"2024-05-09T08:28:57.838704+0000","last_peered":"2024-05-09T08:28:57.838704+0000","last_clean":"2024-05-09T08:28:57.838704+0000","last_became_active":"2024-05-09T08:26:34.965800+0000","last_became_peered":"2024-05-09T08:26:34.965800+0000","last_unstale":"2024-05-09T08:28:57.838704+0000","last_undegraded":"2024-05-09T08:28:57.838704+0000","last_fullsized":"2024-05-09T08:28:57.838704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:22:41.596226+0000","last_clean_scrub_stamp":"2024-05-09T08:22:41.596226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:37:10.419566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.834552+0000","last_change":"2024-05-09T08:27:55.805417+0000","last_active":"2024-05-09T08:28:57.834552+0000","last_peered":"2024-05-09T08:28:57.834552+0000","last_clean":"2024-05-09T08:28:57.834552+0000","last_became_active":"2024-05-09T08:27:55.805025+0000","last_became_peered":"2024-05-09T08:27:55.805025+0000","last_unstale":"2024-05-09T08:28:57.834552+0000","last_undegraded":"2024-05-09T08:28:57.834552+0000","last_fullsized":"2024-05-09T08:28:57.834552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:35.442848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839002+0000","last_change":"2024-05-09T08:27:54.769276+0000","last_active":"2024-05-09T08:28:57.839002+0000","last_peered":"2024-05-09T08:28:57.839002+0000","last_clean":"2024-05-09T08:28:57.839002+0000","last_became_active":"2024-05-09T08:27:54.768650+0000","last_became_peered":"2024-05-09T08:27:54.768650+0000","last_unstale":"2024-05-09T08:28:57.839002+0000","last_undegraded":"2024-05-09T08:28:57.839002+0000","last_fullsized":"2024-05-09T08:28:57.839002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:32.480610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.842994+0000","last_change":"2024-05-09T08:27:54.782809+0000","last_active":"2024-05-09T08:28:57.842994+0000","last_peered":"2024-05-09T08:28:57.842994+0000","last_clean":"2024-05-09T08:28:57.842994+0000","last_became_active":"2024-05-09T08:27:54.782452+0000","last_became_peered":"2024-05-09T08:27:54.782452+0000","last_unstale":"2024-05-09T08:28:57.842994+0000","last_undegraded":"2024-05-09T08:28:57.842994+0000","last_fullsized":"2024-05-09T08:28:57.842994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:42.371573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835639+0000","last_change":"2024-05-09T08:27:55.814683+0000","last_active":"2024-05-09T08:28:57.835639+0000","last_peered":"2024-05-09T08:28:57.835639+0000","last_clean":"2024-05-09T08:28:57.835639+0000","last_became_active":"2024-05-09T08:27:55.813788+0000","last_became_peered":"2024-05-09T08:27:55.813788+0000","last_unstale":"2024-05-09T08:28:57.835639+0000","last_undegraded":"2024-05-09T08:28:57.835639+0000","last_fullsized":"2024-05-09T08:28:57.835639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:45.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.829381+0000","last_change":"2024-05-09T08:27:54.788393+0000","last_active":"2024-05-09T08:28:57.829381+0000","last_peered":"2024-05-09T08:28:57.829381+0000","last_clean":"2024-05-09T08:28:57.829381+0000","last_became_active":"2024-05-09T08:27:54.788031+0000","last_became_peered":"2024-05-09T08:27:54.788031+0000","last_unstale":"2024-05-09T08:28:57.829381+0000","last_undegraded":"2024-05-09T08:28:57.829381+0000","last_fullsized":"2024-05-09T08:28:57.829381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:26.451385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844858+0000","last_change":"2024-05-09T08:27:55.812391+0000","last_active":"2024-05-09T08:28:57.844858+0000","last_peered":"2024-05-09T08:28:57.844858+0000","last_clean":"2024-05-09T08:28:57.844858+0000","last_became_active":"2024-05-09T08:27:55.811954+0000","last_became_peered":"2024-05-09T08:27:55.811954+0000","last_unstale":"2024-05-09T08:28:57.844858+0000","last_undegraded":"2024-05-09T08:28:57.844858+0000","last_fullsized":"2024-05-09T08:28:57.844858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:38.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940042+0000","last_change":"2024-05-09T08:27:54.775693+0000","last_active":"2024-05-09T08:28:57.940042+0000","last_peered":"2024-05-09T08:28:57.940042+0000","last_clean":"2024-05-09T08:28:57.940042+0000","last_became_active":"2024-05-09T08:27:54.775198+0000","last_became_peered":"2024-05-09T08:27:54.775198+0000","last_unstale":"2024-05-09T08:28:57.940042+0000","last_undegraded":"2024-05-09T08:28:57.940042+0000","last_fullsized":"2024-05-09T08:28:57.940042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:21:18.672744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.833562+0000","last_change":"2024-05-09T08:27:55.822832+0000","last_active":"2024-05-09T08:28:57.833562+0000","last_peered":"2024-05-09T08:28:57.833562+0000","last_clean":"2024-05-09T08:28:57.833562+0000","last_became_active":"2024-05-09T08:27:55.822391+0000","last_became_peered":"2024-05-09T08:27:55.822391+0000","last_unstale":"2024-05-09T08:28:57.833562+0000","last_undegraded":"2024-05-09T08:28:57.833562+0000","last_fullsized":"2024-05-09T08:28:57.833562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:15:00.076081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940621+0000","last_change":"2024-05-09T08:27:54.791027+0000","last_active":"2024-05-09T08:28:57.940621+0000","last_peered":"2024-05-09T08:28:57.940621+0000","last_clean":"2024-05-09T08:28:57.940621+0000","last_became_active":"2024-05-09T08:27:54.790726+0000","last_became_peered":"2024-05-09T08:27:54.790726+0000","last_unstale":"2024-05-09T08:28:57.940621+0000","last_undegraded":"2024-05-09T08:28:57.940621+0000","last_fullsized":"2024-05-09T08:28:57.940621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:37.795710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841994+0000","last_change":"2024-05-09T08:27:55.799349+0000","last_active":"2024-05-09T08:28:57.841994+0000","last_peered":"2024-05-09T08:28:57.841994+0000","last_clean":"2024-05-09T08:28:57.841994+0000","last_became_active":"2024-05-09T08:27:55.798931+0000","last_became_peered":"2024-05-09T08:27:55.798931+0000","last_unstale":"2024-05-09T08:28:57.841994+0000","last_undegraded":"2024-05-09T08:28:57.841994+0000","last_fullsized":"2024-05-09T08:28:57.841994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:13:15.824884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843389+0000","last_change":"2024-05-09T08:27:54.792088+0000","last_active":"2024-05-09T08:28:57.843389+0000","last_peered":"2024-05-09T08:28:57.843389+0000","last_clean":"2024-05-09T08:28:57.843389+0000","last_became_active":"2024-05-09T08:27:54.791526+0000","last_became_peered":"2024-05-09T08:27:54.791526+0000","last_unstale":"2024-05-09T08:28:57.843389+0000","last_undegraded":"2024-05-09T08:28:57.843389+0000","last_fullsized":"2024-05-09T08:28:57.843389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:12:11.494625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838345+0000","last_change":"2024-05-09T08:27:55.823258+0000","last_active":"2024-05-09T08:28:57.838345+0000","last_peered":"2024-05-09T08:28:57.838345+0000","last_clean":"2024-05-09T08:28:57.838345+0000","last_became_active":"2024-05-09T08:27:55.822923+0000","last_became_peered":"2024-05-09T08:27:55.822923+0000","last_unstale":"2024-05-09T08:28:57.838345+0000","last_undegraded":"2024-05-09T08:28:57.838345+0000","last_fullsized":"2024-05-09T08:28:57.838345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:02:17.481997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.849289+0000","last_change":"2024-05-09T08:28:57.848915+0000","last_active":"2024-05-09T08:28:57.849289+0000","last_peered":"2024-05-09T08:28:57.849289+0000","last_clean":"2024-05-09T08:28:57.849289+0000","last_became_active":"2024-05-09T08:28:57.844301+0000","last_became_peered":"2024-05-09T08:28:57.844301+0000","last_unstale":"2024-05-09T08:28:57.849289+0000","last_undegraded":"2024-05-09T08:28:57.849289+0000","last_fullsized":"2024-05-09T08:28:57.849289+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:59.620437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843102+0000","last_change":"2024-05-09T08:27:55.829385+0000","last_active":"2024-05-09T08:28:57.843102+0000","last_peered":"2024-05-09T08:28:57.843102+0000","last_clean":"2024-05-09T08:28:57.843102+0000","last_became_active":"2024-05-09T08:27:55.829045+0000","last_became_peered":"2024-05-09T08:27:55.829045+0000","last_unstale":"2024-05-09T08:28:57.843102+0000","last_undegraded":"2024-05-09T08:28:57.843102+0000","last_fullsized":"2024-05-09T08:28:57.843102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.171868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.847090+0000","last_change":"2024-05-09T08:28:57.846755+0000","last_active":"2024-05-09T08:28:57.847090+0000","last_peered":"2024-05-09T08:28:57.847090+0000","last_clean":"2024-05-09T08:28:57.847090+0000","last_became_active":"2024-05-09T08:28:57.841834+0000","last_became_peered":"2024-05-09T08:28:57.841834+0000","last_unstale":"2024-05-09T08:28:57.847090+0000","last_undegraded":"2024-05-09T08:28:57.847090+0000","last_fullsized":"2024-05-09T08:28:57.847090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:39:08.123561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838949+0000","last_change":"2024-05-09T08:27:55.804210+0000","last_active":"2024-05-09T08:28:57.838949+0000","last_peered":"2024-05-09T08:28:57.838949+0000","last_clean":"2024-05-09T08:28:57.838949+0000","last_became_active":"2024-05-09T08:27:55.803870+0000","last_became_peered":"2024-05-09T08:27:55.803870+0000","last_unstale":"2024-05-09T08:28:57.838949+0000","last_undegraded":"2024-05-09T08:28:57.838949+0000","last_fullsized":"2024-05-09T08:28:57.838949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:46.593084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841120+0000","last_change":"2024-05-09T08:27:54.775110+0000","last_active":"2024-05-09T08:28:57.841120+0000","last_peered":"2024-05-09T08:28:57.841120+0000","last_clean":"2024-05-09T08:28:57.841120+0000","last_became_active":"2024-05-09T08:27:54.774812+0000","last_became_peered":"2024-05-09T08:27:54.774812+0000","last_unstale":"2024-05-09T08:28:57.841120+0000","last_undegraded":"2024-05-09T08:28:57.841120+0000","last_fullsized":"2024-05-09T08:28:57.841120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:51.955284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838938+0000","last_change":"2024-05-09T08:27:55.804766+0000","last_active":"2024-05-09T08:28:57.838938+0000","last_peered":"2024-05-09T08:28:57.838938+0000","last_clean":"2024-05-09T08:28:57.838938+0000","last_became_active":"2024-05-09T08:27:55.803711+0000","last_became_peered":"2024-05-09T08:27:55.803711+0000","last_unstale":"2024-05-09T08:28:57.838938+0000","last_undegraded":"2024-05-09T08:28:57.838938+0000","last_fullsized":"2024-05-09T08:28:57.838938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:17:46.683112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838726+0000","last_change":"2024-05-09T08:27:54.786594+0000","last_active":"2024-05-09T08:28:57.838726+0000","last_peered":"2024-05-09T08:28:57.838726+0000","last_clean":"2024-05-09T08:28:57.838726+0000","last_became_active":"2024-05-09T08:27:54.786329+0000","last_became_peered":"2024-05-09T08:27:54.786329+0000","last_unstale":"2024-05-09T08:28:57.838726+0000","last_undegraded":"2024-05-09T08:28:57.838726+0000","last_fullsized":"2024-05-09T08:28:57.838726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:09:43.496319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.849973+0000","last_change":"2024-05-09T08:28:57.849973+0000","last_active":"2024-05-09T08:28:57.849973+0000","last_peered":"2024-05-09T08:28:57.849973+0000","last_clean":"2024-05-09T08:28:57.849973+0000","last_became_active":"2024-05-09T08:28:57.844256+0000","last_became_peered":"2024-05-09T08:28:57.844256+0000","last_unstale":"2024-05-09T08:28:57.849973+0000","last_undegraded":"2024-05-09T08:28:57.849973+0000","last_fullsized":"2024-05-09T08:28:57.849973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:33:56.058933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835083+0000","last_change":"2024-05-09T08:27:54.766960+0000","last_active":"2024-05-09T08:28:57.835083+0000","last_peered":"2024-05-09T08:28:57.835083+0000","last_clean":"2024-05-09T08:28:57.835083+0000","last_became_active":"2024-05-09T08:27:54.765693+0000","last_became_peered":"2024-05-09T08:27:54.765693+0000","last_unstale":"2024-05-09T08:28:57.835083+0000","last_undegraded":"2024-05-09T08:28:57.835083+0000","last_fullsized":"2024-05-09T08:28:57.835083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:53.735347+0000","last_clean_scrub_stamp":"2024-05-09T08:27:53.735347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:29:50.080788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838822+0000","last_change":"2024-05-09T08:27:55.806802+0000","last_active":"2024-05-09T08:28:57.838822+0000","last_peered":"2024-05-09T08:28:57.838822+0000","last_clean":"2024-05-09T08:28:57.838822+0000","last_became_active":"2024-05-09T08:27:55.806421+0000","last_became_peered":"2024-05-09T08:27:55.806421+0000","last_unstale":"2024-05-09T08:28:57.838822+0000","last_undegraded":"2024-05-09T08:28:57.838822+0000","last_fullsized":"2024-05-09T08:28:57.838822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:45.673043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.838373+0000","last_change":"2024-05-09T08:27:55.799360+0000","last_active":"2024-05-09T08:28:57.838373+0000","last_peered":"2024-05-09T08:28:57.838373+0000","last_clean":"2024-05-09T08:28:57.838373+0000","last_became_active":"2024-05-09T08:27:55.799018+0000","last_became_peered":"2024-05-09T08:27:55.799018+0000","last_unstale":"2024-05-09T08:28:57.838373+0000","last_undegraded":"2024-05-09T08:28:57.838373+0000","last_fullsized":"2024-05-09T08:28:57.838373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:54.767546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.840915+0000","last_change":"2024-05-09T08:27:55.803385+0000","last_active":"2024-05-09T08:28:57.840915+0000","last_peered":"2024-05-09T08:28:57.840915+0000","last_clean":"2024-05-09T08:28:57.840915+0000","last_became_active":"2024-05-09T08:27:55.803031+0000","last_became_peered":"2024-05-09T08:27:55.803031+0000","last_unstale":"2024-05-09T08:28:57.840915+0000","last_undegraded":"2024-05-09T08:28:57.840915+0000","last_fullsized":"2024-05-09T08:28:57.840915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:47.750225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841167+0000","last_change":"2024-05-09T08:27:55.825634+0000","last_active":"2024-05-09T08:28:57.841167+0000","last_peered":"2024-05-09T08:28:57.841167+0000","last_clean":"2024-05-09T08:28:57.841167+0000","last_became_active":"2024-05-09T08:27:55.825330+0000","last_became_peered":"2024-05-09T08:27:55.825330+0000","last_unstale":"2024-05-09T08:28:57.841167+0000","last_undegraded":"2024-05-09T08:28:57.841167+0000","last_fullsized":"2024-05-09T08:28:57.841167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:21:18.166324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.832666+0000","last_change":"2024-05-09T08:27:55.818372+0000","last_active":"2024-05-09T08:28:57.832666+0000","last_peered":"2024-05-09T08:28:57.832666+0000","last_clean":"2024-05-09T08:28:57.832666+0000","last_became_active":"2024-05-09T08:27:55.817982+0000","last_became_peered":"2024-05-09T08:27:55.817982+0000","last_unstale":"2024-05-09T08:28:57.832666+0000","last_undegraded":"2024-05-09T08:28:57.832666+0000","last_fullsized":"2024-05-09T08:28:57.832666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:16.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.836616+0000","last_change":"2024-05-09T08:27:55.820509+0000","last_active":"2024-05-09T08:28:57.836616+0000","last_peered":"2024-05-09T08:28:57.836616+0000","last_clean":"2024-05-09T08:28:57.836616+0000","last_became_active":"2024-05-09T08:27:55.820135+0000","last_became_peered":"2024-05-09T08:27:55.820135+0000","last_unstale":"2024-05-09T08:28:57.836616+0000","last_undegraded":"2024-05-09T08:28:57.836616+0000","last_fullsized":"2024-05-09T08:28:57.836616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:42:28.691640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839350+0000","last_change":"2024-05-09T08:27:55.801038+0000","last_active":"2024-05-09T08:28:57.839350+0000","last_peered":"2024-05-09T08:28:57.839350+0000","last_clean":"2024-05-09T08:28:57.839350+0000","last_became_active":"2024-05-09T08:27:55.800700+0000","last_became_peered":"2024-05-09T08:27:55.800700+0000","last_unstale":"2024-05-09T08:28:57.839350+0000","last_undegraded":"2024-05-09T08:28:57.839350+0000","last_fullsized":"2024-05-09T08:28:57.839350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:44:42.978833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.841255+0000","last_change":"2024-05-09T08:27:55.802266+0000","last_active":"2024-05-09T08:28:57.841255+0000","last_peered":"2024-05-09T08:28:57.841255+0000","last_clean":"2024-05-09T08:28:57.841255+0000","last_became_active":"2024-05-09T08:27:55.801670+0000","last_became_peered":"2024-05-09T08:27:55.801670+0000","last_unstale":"2024-05-09T08:28:57.841255+0000","last_undegraded":"2024-05-09T08:28:57.841255+0000","last_fullsized":"2024-05-09T08:28:57.841255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:53.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844724+0000","last_change":"2024-05-09T08:27:55.820107+0000","last_active":"2024-05-09T08:28:57.844724+0000","last_peered":"2024-05-09T08:28:57.844724+0000","last_clean":"2024-05-09T08:28:57.844724+0000","last_became_active":"2024-05-09T08:27:55.819748+0000","last_became_peered":"2024-05-09T08:27:55.819748+0000","last_unstale":"2024-05-09T08:28:57.844724+0000","last_undegraded":"2024-05-09T08:28:57.844724+0000","last_fullsized":"2024-05-09T08:28:57.844724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:08:04.311374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.839635+0000","last_change":"2024-05-09T08:27:55.821625+0000","last_active":"2024-05-09T08:28:57.839635+0000","last_peered":"2024-05-09T08:28:57.839635+0000","last_clean":"2024-05-09T08:28:57.839635+0000","last_became_active":"2024-05-09T08:27:55.821291+0000","last_became_peered":"2024-05-09T08:27:55.821291+0000","last_unstale":"2024-05-09T08:28:57.839635+0000","last_undegraded":"2024-05-09T08:28:57.839635+0000","last_fullsized":"2024-05-09T08:28:57.839635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:40.017767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.843250+0000","last_change":"2024-05-09T08:27:55.818212+0000","last_active":"2024-05-09T08:28:57.843250+0000","last_peered":"2024-05-09T08:28:57.843250+0000","last_clean":"2024-05-09T08:28:57.843250+0000","last_became_active":"2024-05-09T08:27:55.817874+0000","last_became_peered":"2024-05-09T08:27:55.817874+0000","last_unstale":"2024-05-09T08:28:57.843250+0000","last_undegraded":"2024-05-09T08:28:57.843250+0000","last_fullsized":"2024-05-09T08:28:57.843250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:20.189942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.940739+0000","last_change":"2024-05-09T08:27:55.820128+0000","last_active":"2024-05-09T08:28:57.940739+0000","last_peered":"2024-05-09T08:28:57.940739+0000","last_clean":"2024-05-09T08:28:57.940739+0000","last_became_active":"2024-05-09T08:27:55.819717+0000","last_became_peered":"2024-05-09T08:27:55.819717+0000","last_unstale":"2024-05-09T08:28:57.940739+0000","last_undegraded":"2024-05-09T08:28:57.940739+0000","last_fullsized":"2024-05-09T08:28:57.940739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:48.324777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.835270+0000","last_change":"2024-05-09T08:27:55.820639+0000","last_active":"2024-05-09T08:28:57.835270+0000","last_peered":"2024-05-09T08:28:57.835270+0000","last_clean":"2024-05-09T08:28:57.835270+0000","last_became_active":"2024-05-09T08:27:55.820116+0000","last_became_peered":"2024-05-09T08:27:55.820116+0000","last_unstale":"2024-05-09T08:28:57.835270+0000","last_undegraded":"2024-05-09T08:28:57.835270+0000","last_fullsized":"2024-05-09T08:28:57.835270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:29:35.099440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-09T08:28:57.844163+0000","last_change":"2024-05-09T08:28:57.844163+0000","last_active":"2024-05-09T08:28:57.844163+0000","last_peered":"2024-05-09T08:28:57.844163+0000","last_clean":"2024-05-09T08:28:57.844163+0000","last_became_active":"2024-05-09T08:28:57.843753+0000","last_became_peered":"2024-05-09T08:28:57.843753+0000","last_unstale":"2024-05-09T08:28:57.844163+0000","last_undegraded":"2024-05-09T08:28:57.844163+0000","last_fullsized":"2024-05-09T08:28:57.844163+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:27:54.750080+0000","last_clean_scrub_stamp":"2024-05-09T08:27:54.750080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:09:31.238074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280679,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63466,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":531,"num_read_kb":333,"num_write":155,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":190398,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":459740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1111077,"data_compressed":37567,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":459896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":1051826,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":459482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":518623,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":460636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1036288,"data_stored":519973,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":459676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":460193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1051002,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35701,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60815,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61434,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:29:27.737 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T08:29:27.737 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T08:29:27.737 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T08:29:27.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-09T08:29:27.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:27 smithi133 ceph-mon[29610]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:28.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:27 smithi160 ceph-mon[34070]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:28.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:27 smithi188 ceph-mon[34645]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:28.802 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:28 smithi133 ceph-mon[29610]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:28.802 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:28 smithi133 ceph-mon[29610]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T08:29:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:28 smithi160 ceph-mon[34070]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:28 smithi160 ceph-mon[34070]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T08:29:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:28 smithi188 ceph-mon[34645]: from='client.24884 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T08:29:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:28 smithi188 ceph-mon[34645]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T08:29:30.052 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:29 smithi133 ceph-mon[29610]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:29 smithi160 ceph-mon[34070]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:29 smithi188 ceph-mon[34645]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:30.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 -- 172.21.15.133:0/1204571615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 msgr2=0x7fe49010f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:30.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1204571615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49010f150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b0d0 tx=0x7fe48800b3e0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 -- 172.21.15.133:0/1204571615 shutdown_connections 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1204571615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49010f150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1204571615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe490102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1204571615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe490106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 -- 172.21.15.133:0/1204571615 >> 172.21.15.133:0/1204571615 conn(0x7fe4900fdd30 msgr2=0x7fe490100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 -- 172.21.15.133:0/1204571615 shutdown_connections 2024-05-09T08:29:30.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.551+0000 7fe48fe00700 1 -- 172.21.15.133:0/1204571615 wait complete. 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 Processor -- start 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 -- start start 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe4901a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe4901a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe490112590 con 0x7fe490107580 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe490112410 con 0x7fe4901024a0 2024-05-09T08:29:30.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.553+0000 7fe48fe00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe490112710 con 0x7fe490102e50 2024-05-09T08:29:30.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:30.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48ea00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe4901a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe4901a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45984/0 (socket says 172.21.15.133:45984) 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48f400700 1 -- 172.21.15.133:0/1509709563 learned_addr learned my addr 172.21.15.133:0/1509709563 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 -- 172.21.15.133:0/1509709563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 msgr2=0x7fe4901a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe4901a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 -- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 msgr2=0x7fe49019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 -- 172.21.15.133:0/1509709563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49019b0f0 con 0x7fe490102e50 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48f400700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe48ea00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe4901a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.554+0000 7fe486000700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe4901a0d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d720 tx=0x7fe47800da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.555+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478016070 con 0x7fe490102e50 2024-05-09T08:29:30.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.555+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478005020 con 0x7fe490102e50 2024-05-09T08:29:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.555+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49019b3e0 con 0x7fe490102e50 2024-05-09T08:29:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.555+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4901a76a0 con 0x7fe490102e50 2024-05-09T08:29:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.555+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800fb70 con 0x7fe490102e50 2024-05-09T08:29:30.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.556+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe490069010 con 0x7fe490102e50 2024-05-09T08:29:30.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.558+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe4780051d0 con 0x7fe490102e50 2024-05-09T08:29:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.559+0000 7fe487e00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 0x7fe4740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.559+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe4780ce8b0 con 0x7fe490102e50 2024-05-09T08:29:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.560+0000 7fe48ea00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 0x7fe4740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:30.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.560+0000 7fe48ea00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 0x7fe4740a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49019bad0 tx=0x7fe4800073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:30.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.563+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe47809b7e0 con 0x7fe490102e50 2024-05-09T08:29:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.844+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe4901a7930 con 0x7fe490102e50 2024-05-09T08:29:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.845+0000 7fe487e00700 1 -- 172.21.15.133:0/1509709563 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe47809b0e0 con 0x7fe490102e50 2024-05-09T08:29:30.845 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:30.845 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T08:29:30.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 msgr2=0x7fe4740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:30.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 0x7fe4740a5160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49019bad0 tx=0x7fe4800073d0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 msgr2=0x7fe4901a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:30.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe4901a0d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d720 tx=0x7fe47800da30 comp rx=0 tx=0).stop 2024-05-09T08:29:30.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 shutdown_connections 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe490107580 0x7fe49019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4901024a0 0x7fe4901a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe4740a2ca0 0x7fe4740a5160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 --2- 172.21.15.133:0/1509709563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe490102e50 0x7fe4901a0d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 >> 172.21.15.133:0/1509709563 conn(0x7fe4900fdd30 msgr2=0x7fe49010d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 shutdown_connections 2024-05-09T08:29:30.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:30.848+0000 7fe48fe00700 1 -- 172.21.15.133:0/1509709563 wait complete. 2024-05-09T08:29:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:30 smithi160 ceph-mon[34070]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:29:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:30 smithi133 ceph-mon[29610]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:29:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:30 smithi188 ceph-mon[34645]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:29:31.646 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T08:29:31.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 -- 172.21.15.133:0/3747783502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480103ab0 msgr2=0x7f6480103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 --2- 172.21.15.133:0/3747783502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480103ab0 0x7f6480103ed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6470009900 tx=0x7f647002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 -- 172.21.15.133:0/3747783502 shutdown_connections 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 --2- 172.21.15.133:0/3747783502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480105660 0x7f648010bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 --2- 172.21.15.133:0/3747783502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f6480105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 --2- 172.21.15.133:0/3747783502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480103ab0 0x7f6480103ed0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 -- 172.21.15.133:0/3747783502 >> 172.21.15.133:0/3747783502 conn(0x7f64800ff020 msgr2=0x7f6480101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 -- 172.21.15.133:0/3747783502 shutdown_connections 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.862+0000 7f6486600640 1 -- 172.21.15.133:0/3747783502 wait complete. 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.863+0000 7f6486600640 1 Processor -- start 2024-05-09T08:29:31.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.863+0000 7f6486600640 1 -- start start 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480103ab0 0x7f6480199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480105660 0x7f64801a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f648010f700 con 0x7f6480105660 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f648010f580 con 0x7f6480104ca0 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f6486600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f648010f880 con 0x7f6480103ab0 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480105660 0x7f64801a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480103ab0 0x7f6480199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:31.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:56878/0 (socket says 172.21.15.133:56878) 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 -- 172.21.15.133:0/4225099879 learned_addr learned my addr 172.21.15.133:0/4225099879 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 -- 172.21.15.133:0/4225099879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480103ab0 msgr2=0x7f6480199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480103ab0 0x7f6480199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 -- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480105660 msgr2=0x7f64801a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480105660 0x7f64801a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.864+0000 7f647ea00640 1 -- 172.21.15.133:0/4225099879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64801a19f0 con 0x7f6480104ca0 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f647ea00640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e9f0 tx=0x7f646c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00cd80 con 0x7f6480104ca0 2024-05-09T08:29:31.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64801a1ce0 con 0x7f6480104ca0 2024-05-09T08:29:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c010910 con 0x7f6480104ca0 2024-05-09T08:29:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c0188f0 con 0x7f6480104ca0 2024-05-09T08:29:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.865+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64801a21d0 con 0x7f6480104ca0 2024-05-09T08:29:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.866+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648010ae50 con 0x7f6480104ca0 2024-05-09T08:29:31.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.869+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f646c010ac0 con 0x7f6480104ca0 2024-05-09T08:29:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.870+0000 7f6477e00640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 0x7f64640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.870+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f646c0cf420 con 0x7f6480104ca0 2024-05-09T08:29:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.870+0000 7f647f400640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 0x7f64640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:31.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.871+0000 7f647f400640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 0x7f64640a5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6470005ab0 tx=0x7f6470000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:31.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:31.874+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f646c09bd00 con 0x7f6480104ca0 2024-05-09T08:29:31.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:31 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1509709563' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T08:29:32.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.075+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64800624d0 con 0x7f6480104ca0 2024-05-09T08:29:32.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.077+0000 7f6477e00640 1 -- 172.21.15.133:0/4225099879 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7f646c0a0000 con 0x7f6480104ca0 2024-05-09T08:29:32.078 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:32.078 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:29:32.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 msgr2=0x7f64640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 0x7f64640a5050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6470005ab0 tx=0x7f6470000bc0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 msgr2=0x7f648019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e9f0 tx=0x7f646c00eeb0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 shutdown_connections 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f64640a2b90 0x7f64640a5050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6480105660 0x7f64801a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6480104ca0 0x7f648019a310 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 --2- 172.21.15.133:0/4225099879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6480103ab0 0x7f6480199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.080+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 >> 172.21.15.133:0/4225099879 conn(0x7f64800ff020 msgr2=0x7f6480108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.081+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 shutdown_connections 2024-05-09T08:29:32.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.081+0000 7f6486600640 1 -- 172.21.15.133:0/4225099879 wait complete. 2024-05-09T08:29:32.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:29:32.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:29:32.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:31 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1509709563' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T08:29:32.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:31 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1509709563' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.310+0000 7f51c8a00640 1 -- 172.21.15.133:0/1921816542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4104c80 msgr2=0x7f51c4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.310+0000 7f51c8a00640 1 --2- 172.21.15.133:0/1921816542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4104c80 0x7f51c4105100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009900 tx=0x7f51ac02a410 comp rx=0 tx=0).stop 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 -- 172.21.15.133:0/1921816542 shutdown_connections 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 --2- 172.21.15.133:0/1921816542 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4105640 0x7f51c410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 --2- 172.21.15.133:0/1921816542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4104c80 0x7f51c4105100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 --2- 172.21.15.133:0/1921816542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4103a90 0x7f51c4103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 -- 172.21.15.133:0/1921816542 >> 172.21.15.133:0/1921816542 conn(0x7f51c40ff020 msgr2=0x7f51c4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 -- 172.21.15.133:0/1921816542 shutdown_connections 2024-05-09T08:29:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.311+0000 7f51c8a00640 1 -- 172.21.15.133:0/1921816542 wait complete. 2024-05-09T08:29:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 Processor -- start 2024-05-09T08:29:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 -- start start 2024-05-09T08:29:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4104c80 0x7f51c419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410f740 con 0x7f51c4105640 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410f5c0 con 0x7f51c4103a90 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c8a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51c410f8c0 con 0x7f51c4104c80 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c1e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c1e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:56908/0 (socket says 172.21.15.133:56908) 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46024/0 (socket says 172.21.15.133:46024) 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51c1e00640 1 -- 172.21.15.133:0/179475468 learned_addr learned my addr 172.21.15.133:0/179475468 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.312+0000 7f51b9400640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4104c80 0x7f51c419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 -- 172.21.15.133:0/179475468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4104c80 msgr2=0x7f51c419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4104c80 0x7f51c419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 -- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 msgr2=0x7f51c41a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 -- 172.21.15.133:0/179475468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c41a1a30 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c2800640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c1e00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009d20 tx=0x7f51b400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4010630 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b4010cb0 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c41a1d20 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4018ce0 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.313+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c41a2190 con 0x7f51c4103a90 2024-05-09T08:29:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.314+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c410ae00 con 0x7f51c4103a90 2024-05-09T08:29:32.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.317+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f51b401f020 con 0x7f51c4103a90 2024-05-09T08:29:32.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.318+0000 7f51bb400640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 0x7f51a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.318+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f51b40cf730 con 0x7f51c4103a90 2024-05-09T08:29:32.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.318+0000 7f51b9400640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 0x7f51a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.319+0000 7f51b9400640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 0x7f51a80a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51ac031440 tx=0x7f51ac0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:32.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.320+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51b409c660 con 0x7f51c4103a90 2024-05-09T08:29:32.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.519+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51c402d420 con 0x7f51c4103a90 2024-05-09T08:29:32.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.520+0000 7f51bb400640 1 -- 172.21.15.133:0/179475468 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f51b409bf60 con 0x7f51c4103a90 2024-05-09T08:29:32.519 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:29:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.522+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 msgr2=0x7f51a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.522+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 0x7f51a80a5050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51ac031440 tx=0x7f51ac0315b0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.522+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 msgr2=0x7f51c4199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.522+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009d20 tx=0x7f51b400c6a0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 shutdown_connections 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f51a80a2b90 0x7f51a80a5050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c4105640 0x7f51c41a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4104c80 0x7f51c419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 --2- 172.21.15.133:0/179475468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c4103a90 0x7f51c4199e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 >> 172.21.15.133:0/179475468 conn(0x7f51c40ff020 msgr2=0x7f51c4107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 shutdown_connections 2024-05-09T08:29:32.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.523+0000 7f51c8a00640 1 -- 172.21.15.133:0/179475468 wait complete. 2024-05-09T08:29:32.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 -- 172.21.15.133:0/4034746623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 msgr2=0x7f3f00074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 --2- 172.21.15.133:0/4034746623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f00074710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009930 tx=0x7f3ee802a440 comp rx=0 tx=0).stop 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 -- 172.21.15.133:0/4034746623 shutdown_connections 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 --2- 172.21.15.133:0/4034746623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f00110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 --2- 172.21.15.133:0/4034746623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f00074710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 --2- 172.21.15.133:0/4034746623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 0x7f3f00073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 -- 172.21.15.133:0/4034746623 >> 172.21.15.133:0/4034746623 conn(0x7f3f000fcd30 msgr2=0x7f3f000ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 -- 172.21.15.133:0/4034746623 shutdown_connections 2024-05-09T08:29:32.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.738+0000 7f3effe00640 1 -- 172.21.15.133:0/4034746623 wait complete. 2024-05-09T08:29:32.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 Processor -- start 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 -- start start 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 0x7f3f0019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f001a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00113a10 con 0x7f3f00074290 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00113890 con 0x7f3f0010dd90 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.739+0000 7f3effe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00113b90 con 0x7f3f00075870 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46042/0 (socket says 172.21.15.133:46042) 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 -- 172.21.15.133:0/4203811147 learned_addr learned my addr 172.21.15.133:0/4203811147 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3eff400640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f001a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efe000640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 0x7f3f0019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 -- 172.21.15.133:0/4203811147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 msgr2=0x7f3f0019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 0x7f3f0019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 -- 172.21.15.133:0/4203811147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 msgr2=0x7f3f001a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f001a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3efea00640 1 -- 172.21.15.133:0/4203811147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001a5d10 con 0x7f3f00074290 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.740+0000 7f3eff400640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f001a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3efea00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000eb90 tx=0x7f3ef000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004b20 con 0x7f3f00074290 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0004cc0 con 0x7f3f00074290 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0010720 con 0x7f3f00074290 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001a5fa0 con 0x7f3f00074290 2024-05-09T08:29:32.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.741+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001a6410 con 0x7f3f00074290 2024-05-09T08:29:32.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.742+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00103e50 con 0x7f3f00074290 2024-05-09T08:29:32.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.744+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3ef0005020 con 0x7f3f00074290 2024-05-09T08:29:32.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.745+0000 7f3ef7400640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 0x7f3ee40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:32.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.746+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3ef00cf680 con 0x7f3f00074290 2024-05-09T08:29:32.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.746+0000 7f3efe000640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 0x7f3ee40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:32.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.746+0000 7f3efe000640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 0x7f3ee40a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009900 tx=0x7f3ee80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.750+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ef009c5b0 con 0x7f3f00074290 2024-05-09T08:29:32.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.954+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f00062430 con 0x7f3f00074290 2024-05-09T08:29:32.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.954+0000 7f3ef7400640 1 -- 172.21.15.133:0/4203811147 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f3ef009beb0 con 0x7f3f00074290 2024-05-09T08:29:32.954 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 msgr2=0x7f3ee40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 0x7f3ee40a5050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8009900 tx=0x7f3ee80315b0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 msgr2=0x7f3f0019e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000eb90 tx=0x7f3ef000ef90 comp rx=0 tx=0).stop 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 shutdown_connections 2024-05-09T08:29:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3ee40a2b90 0x7f3ee40a5050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f0010dd90 0x7f3f001a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f00075870 0x7f3f0019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 --2- 172.21.15.133:0/4203811147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00074290 0x7f3f0019e0f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 >> 172.21.15.133:0/4203811147 conn(0x7f3f000fcd30 msgr2=0x7f3f00101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 shutdown_connections 2024-05-09T08:29:32.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:32.957+0000 7f3effe00640 1 -- 172.21.15.133:0/4203811147 wait complete. 2024-05-09T08:29:32.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:29:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:32 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4225099879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:32 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/179475468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:33.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:32 smithi160 ceph-mon[34070]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:33.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 -- 172.21.15.133:0/768255134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c100a40 msgr2=0x7fd76c100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 --2- 172.21.15.133:0/768255134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c100a40 0x7fd76c100ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd758009ca0 tx=0x7fd75802a8c0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 -- 172.21.15.133:0/768255134 shutdown_connections 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 --2- 172.21.15.133:0/768255134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c1013e0 0x7fd76c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 --2- 172.21.15.133:0/768255134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c100a40 0x7fd76c100ea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 --2- 172.21.15.133:0/768255134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 -- 172.21.15.133:0/768255134 >> 172.21.15.133:0/768255134 conn(0x7fd76c0fbce0 msgr2=0x7fd76c0fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.174+0000 7fd774000640 1 -- 172.21.15.133:0/768255134 shutdown_connections 2024-05-09T08:29:33.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.175+0000 7fd774000640 1 -- 172.21.15.133:0/768255134 wait complete. 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.175+0000 7fd774000640 1 Processor -- start 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 -- start start 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c100a40 0x7fd76c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c1013e0 0x7fd76c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10f700 con 0x7fd76c1013e0 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10f580 con 0x7fd76c100a40 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd774000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10f880 con 0x7fd76c1000e0 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd771600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd771600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:60294/0 (socket says 172.21.15.133:60294) 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd771600640 1 -- 172.21.15.133:0/1117607389 learned_addr learned my addr 172.21.15.133:0/1117607389 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:33.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd772000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c1013e0 0x7fd76c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.176+0000 7fd770c00640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c100a40 0x7fd76c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 -- 172.21.15.133:0/1117607389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c100a40 msgr2=0x7fd76c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c100a40 0x7fd76c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 -- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c1013e0 msgr2=0x7fd76c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c1013e0 0x7fd76c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 -- 172.21.15.133:0/1117607389 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c1a1990 con 0x7fd76c1000e0 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd771600640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd76400ea60 tx=0x7fd76400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:33.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400cde0 con 0x7fd76c1000e0 2024-05-09T08:29:33.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:32 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4225099879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:33.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:32 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/179475468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:33.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:32 smithi133 ceph-mon[29610]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:33.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7640035f0 con 0x7fd76c1000e0 2024-05-09T08:29:33.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.177+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764010670 con 0x7fd76c1000e0 2024-05-09T08:29:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.178+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c1a1c80 con 0x7fd76c1000e0 2024-05-09T08:29:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.178+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a2150 con 0x7fd76c1000e0 2024-05-09T08:29:33.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.179+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c10afc0 con 0x7fd76c1000e0 2024-05-09T08:29:33.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.181+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd7640037a0 con 0x7fd76c1000e0 2024-05-09T08:29:33.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.182+0000 7fd76a000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 0x7fd7500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.182+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd7640cf890 con 0x7fd76c1000e0 2024-05-09T08:29:33.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.183+0000 7fd770c00640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 0x7fd7500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:32 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4225099879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:33.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:32 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/179475468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:33.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:32 smithi188 ceph-mon[34645]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:33.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.183+0000 7fd770c00640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 0x7fd7500a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd758005e30 tx=0x7fd758005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:33.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.185+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd76401d060 con 0x7fd76c1000e0 2024-05-09T08:29:33.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.384+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd76c02d470 con 0x7fd76c1000e0 2024-05-09T08:29:33.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.385+0000 7fd76a000640 1 -- 172.21.15.133:0/1117607389 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fd76409c7c0 con 0x7fd76c1000e0 2024-05-09T08:29:33.385 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:29:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 msgr2=0x7fd7500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 0x7fd7500a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd758005e30 tx=0x7fd758005f50 comp rx=0 tx=0).stop 2024-05-09T08:29:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 msgr2=0x7fd76c199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd76400ea60 tx=0x7fd76400ef20 comp rx=0 tx=0).stop 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 shutdown_connections 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd7500a2c10 0x7fd7500a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd76c1013e0 0x7fd76c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd76c100a40 0x7fd76c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 --2- 172.21.15.133:0/1117607389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd76c1000e0 0x7fd76c199d70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 >> 172.21.15.133:0/1117607389 conn(0x7fd76c0fbce0 msgr2=0x7fd76c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 shutdown_connections 2024-05-09T08:29:33.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.388+0000 7fd774000640 1 -- 172.21.15.133:0/1117607389 wait complete. 2024-05-09T08:29:33.406 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 -- 172.21.15.133:0/1718418337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 msgr2=0x7f92a8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 --2- 172.21.15.133:0/1718418337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8101c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9294009900 tx=0x7f929402a410 comp rx=0 tx=0).stop 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 -- 172.21.15.133:0/1718418337 shutdown_connections 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 --2- 172.21.15.133:0/1718418337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a8102150 0x7f92a81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 --2- 172.21.15.133:0/1718418337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8101c10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 --2- 172.21.15.133:0/1718418337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a81005a0 0x7f92a81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 -- 172.21.15.133:0/1718418337 >> 172.21.15.133:0/1718418337 conn(0x7f92a80fbb10 msgr2=0x7f92a80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 -- 172.21.15.133:0/1718418337 shutdown_connections 2024-05-09T08:29:33.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.608+0000 7f92ae800640 1 -- 172.21.15.133:0/1718418337 wait complete. 2024-05-09T08:29:33.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.609+0000 7f92ae800640 1 Processor -- start 2024-05-09T08:29:33.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.609+0000 7f92ae800640 1 -- start start 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.609+0000 7f92ae800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ae800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ae800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a8102150 0x7f92a81030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ae800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810c270 con 0x7f92a8101790 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ae800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810c0f0 con 0x7f92a81005a0 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ae800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92a810c3f0 con 0x7f92a8102150 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92aca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92aca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46078/0 (socket says 172.21.15.133:46078) 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92aca00640 1 -- 172.21.15.133:0/1705599464 learned_addr learned my addr 172.21.15.133:0/1705599464 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:56952/0 (socket says 172.21.15.133:56952) 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ade00640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a8102150 0x7f92a81030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 -- 172.21.15.133:0/1705599464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a8102150 msgr2=0x7f92a81030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a8102150 0x7f92a81030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 -- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 msgr2=0x7f92a8102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92ad400640 1 -- 172.21.15.133:0/1705599464 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a8103940 con 0x7f92a81005a0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.610+0000 7f92aca00640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92ad400640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f929c00e9c0 tx=0x7f929c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c00ccc0 con 0x7f92a81005a0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a81a1b30 con 0x7f92a81005a0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c0035f0 con 0x7f92a81005a0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c00ce60 con 0x7f92a81005a0 2024-05-09T08:29:33.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.611+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a81a2030 con 0x7f92a81005a0 2024-05-09T08:29:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.612+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a81a2200 con 0x7f92a81005a0 2024-05-09T08:29:33.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.614+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f929c0037a0 con 0x7f92a81005a0 2024-05-09T08:29:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.615+0000 7f92a6000640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 0x7f92900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.616+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f929c0cf580 con 0x7f92a81005a0 2024-05-09T08:29:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.616+0000 7f92aca00640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 0x7f92900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:33.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.616+0000 7f92aca00640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 0x7f92900a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92a81042a0 tx=0x7f92940315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:33.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.618+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f929c09c4b0 con 0x7f92a81005a0 2024-05-09T08:29:33.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.858+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92a81a23d0 con 0x7f92a81005a0 2024-05-09T08:29:33.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.859+0000 7f92a6000640 1 -- 172.21.15.133:0/1705599464 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7f929c09bdb0 con 0x7f92a81005a0 2024-05-09T08:29:33.859 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:33.859 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:29:33.859 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:29:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.861+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 msgr2=0x7f92900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.861+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 0x7f92900a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92a81042a0 tx=0x7f92940315b0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.861+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 msgr2=0x7f92a8104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.861+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f929c00e9c0 tx=0x7f929c00ee80 comp rx=0 tx=0).stop 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 shutdown_connections 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f92900a2b90 0x7f92900a5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a8102150 0x7f92a81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a8101790 0x7f92a8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 --2- 172.21.15.133:0/1705599464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92a81005a0 0x7f92a8104530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 >> 172.21.15.133:0/1705599464 conn(0x7f92a80fbb10 msgr2=0x7f92a80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 shutdown_connections 2024-05-09T08:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:33.862+0000 7f92ae800640 1 -- 172.21.15.133:0/1705599464 wait complete. 2024-05-09T08:29:33.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:29:34.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 -- 172.21.15.133:0/661433323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8108ea0 msgr2=0x7fc3e8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 --2- 172.21.15.133:0/661433323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8108ea0 0x7fc3e8109320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009900 tx=0x7fc3d002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 -- 172.21.15.133:0/661433323 shutdown_connections 2024-05-09T08:29:34.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 --2- 172.21.15.133:0/661433323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8109860 0x7fc3e81100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 --2- 172.21.15.133:0/661433323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8108ea0 0x7fc3e8109320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 --2- 172.21.15.133:0/661433323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8107cb0 0x7fc3e81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.079+0000 7fc3eda00640 1 -- 172.21.15.133:0/661433323 >> 172.21.15.133:0/661433323 conn(0x7fc3e8076c70 msgr2=0x7fc3e8079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:34.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.080+0000 7fc3eda00640 1 -- 172.21.15.133:0/661433323 shutdown_connections 2024-05-09T08:29:34.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.080+0000 7fc3eda00640 1 -- 172.21.15.133:0/661433323 wait complete. 2024-05-09T08:29:34.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.080+0000 7fc3eda00640 1 Processor -- start 2024-05-09T08:29:34.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 -- start start 2024-05-09T08:29:34.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 0x7fc3e819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8109860 0x7fc3e81a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8113910 con 0x7fc3e8108ea0 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8113790 con 0x7fc3e8109860 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3eda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8113a90 con 0x7fc3e8107cb0 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40970/0 (socket says 172.21.15.133:40970) 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3df400640 1 -- 172.21.15.133:0/1293460645 learned_addr learned my addr 172.21.15.133:0/1293460645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3e7e00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 0x7fc3e819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.081+0000 7fc3ed000640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8109860 0x7fc3e81a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 -- 172.21.15.133:0/1293460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 msgr2=0x7fc3e819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 0x7fc3e819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 -- 172.21.15.133:0/1293460645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8109860 msgr2=0x7fc3e81a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8109860 0x7fc3e81a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 -- 172.21.15.133:0/1293460645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e81a5ba0 con 0x7fc3e8108ea0 2024-05-09T08:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3e7e00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 0x7fc3e819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3df400640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009230 tx=0x7fc3d0031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d003f030 con 0x7fc3e8108ea0 2024-05-09T08:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d0003ad0 con 0x7fc3e8108ea0 2024-05-09T08:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e81a5e30 con 0x7fc3e8108ea0 2024-05-09T08:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0035070 con 0x7fc3e8108ea0 2024-05-09T08:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.082+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a6290 con 0x7fc3e8108ea0 2024-05-09T08:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.084+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e810efd0 con 0x7fc3e8108ea0 2024-05-09T08:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.085+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc3d0042030 con 0x7fc3e8108ea0 2024-05-09T08:29:34.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.087+0000 7fc3e5600640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 0x7fc3c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.087+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc3d00ef540 con 0x7fc3e8108ea0 2024-05-09T08:29:34.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.087+0000 7fc3e7e00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 0x7fc3c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.088+0000 7fc3e7e00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 0x7fc3c80a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8108d00 tx=0x7fc3d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.091+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc3d00bbe20 con 0x7fc3e8108ea0 2024-05-09T08:29:34.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:33 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4203811147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:33 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1117607389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:33 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4203811147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:33 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1117607389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:33 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4203811147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:33 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1117607389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:34.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.295+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3e80624d0 con 0x7fc3e8108ea0 2024-05-09T08:29:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.297+0000 7fc3e5600640 1 -- 172.21.15.133:0/1293460645 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7fc3d00c0120 con 0x7fc3e8108ea0 2024-05-09T08:29:34.298 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:34.298 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 msgr2=0x7fc3c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 0x7fc3c80a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8108d00 tx=0x7fc3d800b390 comp rx=0 tx=0).stop 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 msgr2=0x7fc3e819e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009230 tx=0x7fc3d0031b60 comp rx=0 tx=0).stop 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 shutdown_connections 2024-05-09T08:29:34.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc3c80a2b90 0x7fc3c80a5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3e8109860 0x7fc3e81a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3e8108ea0 0x7fc3e819e4c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 --2- 172.21.15.133:0/1293460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3e8107cb0 0x7fc3e819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.300+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 >> 172.21.15.133:0/1293460645 conn(0x7fc3e8076c70 msgr2=0x7fc3e810c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.301+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 shutdown_connections 2024-05-09T08:29:34.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.301+0000 7fc3eda00640 1 -- 172.21.15.133:0/1293460645 wait complete. 2024-05-09T08:29:34.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:29:34.321 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 -- 172.21.15.133:0/2943287856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc074290 msgr2=0x7f22dc074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 --2- 172.21.15.133:0/2943287856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc074290 0x7f22dc074710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009860 tx=0x7f22c802a340 comp rx=0 tx=0).stop 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 -- 172.21.15.133:0/2943287856 shutdown_connections 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 --2- 172.21.15.133:0/2943287856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc10de40 0x7f22dc110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 --2- 172.21.15.133:0/2943287856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc074290 0x7f22dc074710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 --2- 172.21.15.133:0/2943287856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 -- 172.21.15.133:0/2943287856 >> 172.21.15.133:0/2943287856 conn(0x7f22dc0fcde0 msgr2=0x7f22dc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 -- 172.21.15.133:0/2943287856 shutdown_connections 2024-05-09T08:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.523+0000 7f22e2c00640 1 -- 172.21.15.133:0/2943287856 wait complete. 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.524+0000 7f22e2c00640 1 Processor -- start 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.524+0000 7f22e2c00640 1 -- start start 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc074290 0x7f22dc19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc10de40 0x7f22dc1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc113ae0 con 0x7f22dc075870 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc113960 con 0x7f22dc074290 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc113c60 con 0x7f22dc10de40 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40988/0 (socket says 172.21.15.133:40988) 2024-05-09T08:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 -- 172.21.15.133:0/1504883500 learned_addr learned my addr 172.21.15.133:0/1504883500 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22dbe00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc074290 0x7f22dc19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22e0c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc10de40 0x7f22dc1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 -- 172.21.15.133:0/1504883500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc10de40 msgr2=0x7f22dc1a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc10de40 0x7f22dc1a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 -- 172.21.15.133:0/1504883500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc074290 msgr2=0x7f22dc19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc074290 0x7f22dc19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.525+0000 7f22db400640 1 -- 172.21.15.133:0/1504883500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1a5dd0 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22db400640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22c80304b0 tx=0x7f22c8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8037040 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c8032770 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a6060 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80046c0 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.526+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a65a0 con 0x7f22dc075870 2024-05-09T08:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.528+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc104010 con 0x7f22dc075870 2024-05-09T08:29:34.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.530+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f22c8004050 con 0x7f22dc075870 2024-05-09T08:29:34.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.531+0000 7f22d8c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 0x7f22c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.531+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f22c80eec90 con 0x7f22dc075870 2024-05-09T08:29:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.531+0000 7f22dbe00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 0x7f22c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.532+0000 7f22dbe00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 0x7f22c00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f22cc005fd0 tx=0x7f22cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.536+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f22c8049020 con 0x7f22dc075870 2024-05-09T08:29:34.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.736+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22dc062430 con 0x7f22dc075870 2024-05-09T08:29:34.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.737+0000 7f22d8c00640 1 -- 172.21.15.133:0/1504883500 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f22c80bbbc0 con 0x7f22dc075870 2024-05-09T08:29:34.737 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:29:34.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.739+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 msgr2=0x7f22c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.739+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 0x7f22c00a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f22cc005fd0 tx=0x7f22cc009260 comp rx=0 tx=0).stop 2024-05-09T08:29:34.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 msgr2=0x7f22dc19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22c80304b0 tx=0x7f22c8033d90 comp rx=0 tx=0).stop 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 shutdown_connections 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f22c00a2c10 0x7f22c00a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22dc10de40 0x7f22dc1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc075870 0x7f22dc19e6f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 --2- 172.21.15.133:0/1504883500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22dc074290 0x7f22dc19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 >> 172.21.15.133:0/1504883500 conn(0x7f22dc0fcde0 msgr2=0x7f22dc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:34.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 shutdown_connections 2024-05-09T08:29:34.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.740+0000 7f22e2c00640 1 -- 172.21.15.133:0/1504883500 wait complete. 2024-05-09T08:29:34.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 -- 172.21.15.133:0/3022727768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0104ca0 msgr2=0x7feae0105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 --2- 172.21.15.133:0/3022727768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0104ca0 0x7feae0105120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feacc009ca0 tx=0x7feacc02a8c0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 -- 172.21.15.133:0/3022727768 shutdown_connections 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 --2- 172.21.15.133:0/3022727768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0105660 0x7feae010bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 --2- 172.21.15.133:0/3022727768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0104ca0 0x7feae0105120 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 --2- 172.21.15.133:0/3022727768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 -- 172.21.15.133:0/3022727768 >> 172.21.15.133:0/3022727768 conn(0x7feae00ff020 msgr2=0x7feae0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 -- 172.21.15.133:0/3022727768 shutdown_connections 2024-05-09T08:29:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.966+0000 7feae8600640 1 -- 172.21.15.133:0/3022727768 wait complete. 2024-05-09T08:29:34.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 Processor -- start 2024-05-09T08:29:34.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 -- start start 2024-05-09T08:29:34.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0104ca0 0x7feae019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 0x7feae01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feae010f730 con 0x7feae0103ab0 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feae010f5b0 con 0x7feae0105660 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae8600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feae010f8b0 con 0x7feae0104ca0 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.967+0000 7feae5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41002/0 (socket says 172.21.15.133:41002) 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 -- 172.21.15.133:0/1532087456 learned_addr learned my addr 172.21.15.133:0/1532087456 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae6600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 0x7feae01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5200640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0104ca0 0x7feae019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 -- 172.21.15.133:0/1532087456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0104ca0 msgr2=0x7feae019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0104ca0 0x7feae019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 -- 172.21.15.133:0/1532087456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 msgr2=0x7feae01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 0x7feae01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 -- 172.21.15.133:0/1532087456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae01a19b0 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae6600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 0x7feae01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.968+0000 7feae5c00640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fead800e9c0 tx=0x7fead800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.969+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead800cd80 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.969+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead80035f0 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.969+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead8010690 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.969+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae01a1ca0 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.969+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae01a21a0 con 0x7feae0103ab0 2024-05-09T08:29:34.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.970+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae010ae50 con 0x7feae0103ab0 2024-05-09T08:29:34.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.972+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fead80037a0 con 0x7feae0103ab0 2024-05-09T08:29:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.973+0000 7feade000640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.974+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fead80cf4d0 con 0x7feae0103ab0 2024-05-09T08:29:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.974+0000 7feae5200640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.975+0000 7feae5200640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 0x7feac40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feacc006010 tx=0x7feacc000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:34.978+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fead809c400 con 0x7feae0103ab0 2024-05-09T08:29:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:34 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1705599464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:34 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1293460645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:34 smithi160 ceph-mon[34070]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:35.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:34 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1504883500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:35.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.177+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feae0062500 con 0x7feae0103ab0 2024-05-09T08:29:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:34 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1705599464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:34 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1293460645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:34 smithi133 ceph-mon[29610]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:34 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1504883500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:35.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.178+0000 7feade000640 1 -- 172.21.15.133:0/1532087456 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fead809bd00 con 0x7feae0103ab0 2024-05-09T08:29:35.178 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:29:35.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 msgr2=0x7feac40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 0x7feac40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feacc006010 tx=0x7feacc000f40 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 msgr2=0x7feae0199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fead800e9c0 tx=0x7fead800ee80 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 shutdown_connections 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7feac40a2c10 0x7feac40a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feae0105660 0x7feae01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feae0104ca0 0x7feae019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 --2- 172.21.15.133:0/1532087456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae0103ab0 0x7feae0199da0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 >> 172.21.15.133:0/1532087456 conn(0x7feae00ff020 msgr2=0x7feae0108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 shutdown_connections 2024-05-09T08:29:35.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.181+0000 7feae8600640 1 -- 172.21.15.133:0/1532087456 wait complete. 2024-05-09T08:29:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:34 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1705599464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:34 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1293460645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:34 smithi188 ceph-mon[34645]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:34 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1504883500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:29:35.200 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 -- 172.21.15.133:0/1004331720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4109920 msgr2=0x7f5dc41101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1004331720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4109920 0x7f5dc41101b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009ca0 tx=0x7f5dbc02a8c0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 -- 172.21.15.133:0/1004331720 shutdown_connections 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1004331720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4109920 0x7f5dc41101b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1004331720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4108f60 0x7f5dc41093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1004331720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4107d70 0x7f5dc4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 -- 172.21.15.133:0/1004331720 >> 172.21.15.133:0/1004331720 conn(0x7f5dc4076d00 msgr2=0x7f5dc4079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 -- 172.21.15.133:0/1004331720 shutdown_connections 2024-05-09T08:29:35.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.411+0000 7f5dc9000640 1 -- 172.21.15.133:0/1004331720 wait complete. 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.412+0000 7f5dc9000640 1 Processor -- start 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.412+0000 7f5dc9000640 1 -- start start 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 0x7f5dc419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4109920 0x7f5dc41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc4113a40 con 0x7f5dc4108f60 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc41138c0 con 0x7f5dc4109920 2024-05-09T08:29:35.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc9000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc4113bc0 con 0x7f5dc4107d70 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 0x7f5dc419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc3400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:48934/0 (socket says 172.21.15.133:48934) 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc3400640 1 -- 172.21.15.133:0/1531604323 learned_addr learned my addr 172.21.15.133:0/1531604323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc3e00640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4109920 0x7f5dc41a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 -- 172.21.15.133:0/1531604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 msgr2=0x7f5dc419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 -- 172.21.15.133:0/1531604323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4109920 msgr2=0x7f5dc41a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4109920 0x7f5dc41a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dbaa00640 1 -- 172.21.15.133:0/1531604323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41a5be0 con 0x7f5dc4108f60 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.413+0000 7f5dc3400640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dbaa00640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 0x7f5dc419e500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e9f0 tx=0x7f5dac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:35.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00ccc0 con 0x7f5dc4108f60 2024-05-09T08:29:35.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac005020 con 0x7f5dc4108f60 2024-05-09T08:29:35.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0049e0 con 0x7f5dc4108f60 2024-05-09T08:29:35.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc41a5ed0 con 0x7f5dc4108f60 2024-05-09T08:29:35.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.414+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc41a6340 con 0x7f5dc4108f60 2024-05-09T08:29:35.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.415+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc410efc0 con 0x7f5dc4108f60 2024-05-09T08:29:35.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.417+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5dac005560 con 0x7f5dc4108f60 2024-05-09T08:29:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.419+0000 7f5dc0c00640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 0x7f5da80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.419+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5dac0ceeb0 con 0x7f5dc4108f60 2024-05-09T08:29:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.419+0000 7f5dc3400640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 0x7f5da80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.420+0000 7f5dc3400640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 0x7f5da80a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5db4005fd0 tx=0x7f5db400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:35.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.423+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5dac0051d0 con 0x7f5dc4108f60 2024-05-09T08:29:35.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.621+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5dc4062430 con 0x7f5dc4108f60 2024-05-09T08:29:35.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.622+0000 7f5dc0c00640 1 -- 172.21.15.133:0/1531604323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f5dac09bde0 con 0x7f5dc4108f60 2024-05-09T08:29:35.622 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 msgr2=0x7f5da80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 0x7f5da80a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5db4005fd0 tx=0x7f5db400b4a0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 msgr2=0x7f5dc419e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 0x7f5dc419e500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00e9f0 tx=0x7f5dac00eeb0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 shutdown_connections 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5da80a2c10 0x7f5da80a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dc4109920 0x7f5dc41a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dc4108f60 0x7f5dc419e500 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 --2- 172.21.15.133:0/1531604323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc4107d70 0x7f5dc419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 >> 172.21.15.133:0/1531604323 conn(0x7f5dc4076d00 msgr2=0x7f5dc410c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:35.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 shutdown_connections 2024-05-09T08:29:35.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.625+0000 7f5dc9000640 1 -- 172.21.15.133:0/1531604323 wait complete. 2024-05-09T08:29:35.644 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:29:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 -- 172.21.15.133:0/1510433703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d841099a0 msgr2=0x7f4d841102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/1510433703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d841099a0 0x7f4d841102c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009be0 tx=0x7f4d7c02a6c0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 -- 172.21.15.133:0/1510433703 shutdown_connections 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/1510433703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d841099a0 0x7f4d841102c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/1510433703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d84109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/1510433703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d84107e90 0x7f4d841082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 -- 172.21.15.133:0/1510433703 >> 172.21.15.133:0/1510433703 conn(0x7f4d84076e50 msgr2=0x7f4d84079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.846+0000 7f4d8bc00640 1 -- 172.21.15.133:0/1510433703 shutdown_connections 2024-05-09T08:29:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.847+0000 7f4d8bc00640 1 -- 172.21.15.133:0/1510433703 wait complete. 2024-05-09T08:29:35.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 Processor -- start 2024-05-09T08:29:35.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 -- start start 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d8419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d841099a0 0x7f4d841a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84113b40 con 0x7f4d84108fe0 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841139c0 con 0x7f4d84107e90 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d8bc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84113cc0 con 0x7f4d841099a0 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.848+0000 7f4d83e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d8419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:36464/0 (socket says 172.21.15.133:36464) 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 -- 172.21.15.133:0/2965896727 learned_addr learned my addr 172.21.15.133:0/2965896727 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89c00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d841099a0 0x7f4d841a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 -- 172.21.15.133:0/2965896727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d841099a0 msgr2=0x7f4d841a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d841099a0 0x7f4d841a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 -- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 msgr2=0x7f4d8419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d8419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 -- 172.21.15.133:0/2965896727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d841a5da0 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d83e00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d8419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d89200640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.849+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cd80 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.850+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78005500 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.850+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78010690 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.850+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841a6090 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.850+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a65e0 con 0x7f4d84107e90 2024-05-09T08:29:35.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.851+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8410f0d0 con 0x7f4d84107e90 2024-05-09T08:29:35.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.853+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4d780046c0 con 0x7f4d84107e90 2024-05-09T08:29:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.854+0000 7f4d81600640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 0x7f4d6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.855+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4d780cf3f0 con 0x7f4d84107e90 2024-05-09T08:29:35.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.855+0000 7f4d83e00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 0x7f4d6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:35.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.856+0000 7f4d83e00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 0x7f4d6c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d700096c0 tx=0x7f4d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:35.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:35.859+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d7809c320 con 0x7f4d84107e90 2024-05-09T08:29:35.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:35 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1532087456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:35.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:35 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1531604323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:36.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.099+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d84062430 con 0x7f4d84107e90 2024-05-09T08:29:36.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.100+0000 7f4d81600640 1 -- 172.21.15.133:0/2965896727 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7f4d78086070 con 0x7f4d84107e90 2024-05-09T08:29:36.100 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:36.100 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:29:36.100 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 msgr2=0x7f4d6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 0x7f4d6c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d700096c0 tx=0x7f4d70009260 comp rx=0 tx=0).stop 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 msgr2=0x7f4d8419e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e9f0 tx=0x7f4d7800edf0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 shutdown_connections 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4d6c0a2c10 0x7f4d6c0a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d841099a0 0x7f4d841a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d84108fe0 0x7f4d8419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 --2- 172.21.15.133:0/2965896727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4d84107e90 0x7f4d8419e180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.103+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 >> 172.21.15.133:0/2965896727 conn(0x7f4d84076e50 msgr2=0x7f4d8410c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.104+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 shutdown_connections 2024-05-09T08:29:36.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.104+0000 7f4d8bc00640 1 -- 172.21.15.133:0/2965896727 wait complete. 2024-05-09T08:29:36.122 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:29:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:35 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1532087456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:35 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1531604323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:35 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1532087456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:29:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:35 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1531604323' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:29:36.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 -- 172.21.15.133:0/778867034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb8074290 msgr2=0x7efdb8074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 --2- 172.21.15.133:0/778867034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb8074290 0x7efdb8074710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efda0009930 tx=0x7efda002a420 comp rx=0 tx=0).stop 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 -- 172.21.15.133:0/778867034 shutdown_connections 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 --2- 172.21.15.133:0/778867034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb810dca0 0x7efdb8110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 --2- 172.21.15.133:0/778867034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb8074290 0x7efdb8074710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 --2- 172.21.15.133:0/778867034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb8073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 -- 172.21.15.133:0/778867034 >> 172.21.15.133:0/778867034 conn(0x7efdb80fcc40 msgr2=0x7efdb80ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 -- 172.21.15.133:0/778867034 shutdown_connections 2024-05-09T08:29:36.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.323+0000 7efdb7200640 1 -- 172.21.15.133:0/778867034 wait complete. 2024-05-09T08:29:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.324+0000 7efdb7200640 1 Processor -- start 2024-05-09T08:29:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 -- start start 2024-05-09T08:29:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 0x7efdb819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb810dca0 0x7efdb81a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113990 con 0x7efdb8075870 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113810 con 0x7efdb810dca0 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8113b10 con 0x7efdb8074290 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb5e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 0x7efdb819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41080/0 (socket says 172.21.15.133:41080) 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb5400640 1 -- 172.21.15.133:0/2902869388 learned_addr learned my addr 172.21.15.133:0/2902869388 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.325+0000 7efdb6800640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb810dca0 0x7efdb81a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 -- 172.21.15.133:0/2902869388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb810dca0 msgr2=0x7efdb81a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb810dca0 0x7efdb81a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 -- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 msgr2=0x7efdb819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 -- 172.21.15.133:0/2902869388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb81a5c20 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb5e00640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 0x7efdb819e000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efda800e9f0 tx=0x7efda800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda800cd80 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb81a5f10 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda80035f0 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8010670 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.326+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a6380 con 0x7efdb8074290 2024-05-09T08:29:36.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.327+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb8103e70 con 0x7efdb8074290 2024-05-09T08:29:36.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.329+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efda80037a0 con 0x7efdb8074290 2024-05-09T08:29:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.331+0000 7efdaea00640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 0x7efd9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.331+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7efda80cf210 con 0x7efdb8074290 2024-05-09T08:29:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.331+0000 7efdb5400640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 0x7efd9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.332+0000 7efdb5400640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 0x7efd9c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda0000c00 tx=0x7efda00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:36.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.333+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efda809c140 con 0x7efdb8074290 2024-05-09T08:29:36.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.536+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdb802d450 con 0x7efdb8074290 2024-05-09T08:29:36.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.538+0000 7efdaea00640 1 -- 172.21.15.133:0/2902869388 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7efda809ba40 con 0x7efdb8074290 2024-05-09T08:29:36.538 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:36.538 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:29:36.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.540+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 msgr2=0x7efd9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.540+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 0x7efd9c0a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efda0000c00 tx=0x7efda00315b0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.540+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 msgr2=0x7efdb819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 0x7efdb819e000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efda800e9f0 tx=0x7efda800eeb0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 shutdown_connections 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7efd9c0a2c10 0x7efd9c0a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdb810dca0 0x7efdb81a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb8075870 0x7efdb819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 --2- 172.21.15.133:0/2902869388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efdb8074290 0x7efdb819e000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 >> 172.21.15.133:0/2902869388 conn(0x7efdb80fcc40 msgr2=0x7efdb8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 shutdown_connections 2024-05-09T08:29:36.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.541+0000 7efdb7200640 1 -- 172.21.15.133:0/2902869388 wait complete. 2024-05-09T08:29:36.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:29:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 -- 172.21.15.133:0/2130275216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0104ca0 msgr2=0x7fbad0105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 --2- 172.21.15.133:0/2130275216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0104ca0 0x7fbad0105120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009860 tx=0x7fbabc02a340 comp rx=0 tx=0).stop 2024-05-09T08:29:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 -- 172.21.15.133:0/2130275216 shutdown_connections 2024-05-09T08:29:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 --2- 172.21.15.133:0/2130275216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0105660 0x7fbad010bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 --2- 172.21.15.133:0/2130275216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0104ca0 0x7fbad0105120 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 --2- 172.21.15.133:0/2130275216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0103ab0 0x7fbad0103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 -- 172.21.15.133:0/2130275216 >> 172.21.15.133:0/2130275216 conn(0x7fbad00ff020 msgr2=0x7fbad0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 -- 172.21.15.133:0/2130275216 shutdown_connections 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.755+0000 7fbad6a00640 1 -- 172.21.15.133:0/2130275216 wait complete. 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 Processor -- start 2024-05-09T08:29:36.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 -- start start 2024-05-09T08:29:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 0x7fbad0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0105660 0x7fbad01a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010f710 con 0x7fbad0104ca0 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010f590 con 0x7fbad0103ab0 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbad6a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbad010f890 con 0x7fbad0105660 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbacf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbacfe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 0x7fbad0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbacf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41096/0 (socket says 172.21.15.133:41096) 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.756+0000 7fbacf400640 1 -- 172.21.15.133:0/845508951 learned_addr learned my addr 172.21.15.133:0/845508951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbad4a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0105660 0x7fbad01a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 -- 172.21.15.133:0/845508951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0105660 msgr2=0x7fbad01a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0105660 0x7fbad01a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 -- 172.21.15.133:0/845508951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 msgr2=0x7fbad0199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 0x7fbad0199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:36.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 -- 172.21.15.133:0/845508951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad01a19d0 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacfe00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 0x7fbad0199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbacf400640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbabc005940 tx=0x7fbabc033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc03f020 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbabc030850 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad01a1c60 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.757+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc037040 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.758+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a2150 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.759+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad010add0 con 0x7fbad0104ca0 2024-05-09T08:29:36.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.760+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbabc042020 con 0x7fbad0104ca0 2024-05-09T08:29:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.762+0000 7fbaccc00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 0x7fbab40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.762+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fbabc0ef360 con 0x7fbad0104ca0 2024-05-09T08:29:36.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.762+0000 7fbacfe00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 0x7fbab40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:36.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.763+0000 7fbacfe00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 0x7fbab40a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbad0104b00 tx=0x7fbac0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:36.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:36.766+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbabc049020 con 0x7fbad0104ca0 2024-05-09T08:29:37.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.007+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbad0062460 con 0x7fbad0104ca0 2024-05-09T08:29:37.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.009+0000 7fbaccc00640 1 -- 172.21.15.133:0/845508951 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7fbabc0bc2f0 con 0x7fbad0104ca0 2024-05-09T08:29:37.009 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:37.009 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:29:37.010 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 msgr2=0x7fbab40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 0x7fbab40a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbad0104b00 tx=0x7fbac0005f90 comp rx=0 tx=0).stop 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 msgr2=0x7fbad019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbabc005940 tx=0x7fbabc033d90 comp rx=0 tx=0).stop 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 shutdown_connections 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fbab40a2b90 0x7fbab40a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad0105660 0x7fbad01a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0104ca0 0x7fbad019a2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 --2- 172.21.15.133:0/845508951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbad0103ab0 0x7fbad0199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 >> 172.21.15.133:0/845508951 conn(0x7fbad00ff020 msgr2=0x7fbad0107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 shutdown_connections 2024-05-09T08:29:37.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.012+0000 7fbad6a00640 1 -- 172.21.15.133:0/845508951 wait complete. 2024-05-09T08:29:37.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T08:27:55.241118+0000', 'modified': '2024-05-09T08:28:01.270664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24556, 'mds_2': 24536}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6837/3112088995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 3112088995}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3112088995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6837/1342622403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 1342622403}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 1342622403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6839/2528648460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 2528648460}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 2528648460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/3807326844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 3807326844}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 3807326844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.133:6841/2374052560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6840', 'nonce': 2374052560}, {'type': 'v1', 'addr': '172.21.15.133:6841', 'nonce': 2374052560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/145157679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 145157679}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 145157679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24536, 24556, 24566]} 2024-05-09T08:29:37.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T08:29:37.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-09T08:29:37.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:36 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2965896727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:36 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2902869388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.100 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:36 smithi133 ceph-mon[29610]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:36 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2965896727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:36 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2902869388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:36 smithi160 ceph-mon[34070]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:36 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2965896727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:36 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2902869388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:29:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:36 smithi188 ceph-mon[34645]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:37.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 -- 172.21.15.133:0/1119518435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 msgr2=0x7f5568101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 --2- 172.21.15.133:0/1119518435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568101ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f555c009900 tx=0x7f555c02a410 comp rx=0 tx=0).stop 2024-05-09T08:29:37.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 -- 172.21.15.133:0/1119518435 shutdown_connections 2024-05-09T08:29:37.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 --2- 172.21.15.133:0/1119518435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 0x7f5568109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 --2- 172.21.15.133:0/1119518435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f5568102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 --2- 172.21.15.133:0/1119518435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568101ca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.233+0000 7f5570800640 1 -- 172.21.15.133:0/1119518435 >> 172.21.15.133:0/1119518435 conn(0x7f55680fce10 msgr2=0x7f55680ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.234+0000 7f5570800640 1 -- 172.21.15.133:0/1119518435 shutdown_connections 2024-05-09T08:29:37.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.234+0000 7f5570800640 1 -- 172.21.15.133:0/1119518435 wait complete. 2024-05-09T08:29:37.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.234+0000 7f5570800640 1 Processor -- start 2024-05-09T08:29:37.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 -- start start 2024-05-09T08:29:37.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 0x7f55681a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f556810d4d0 con 0x7f5568101880 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f556810d350 con 0x7f5568102a70 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f5570800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f556810d650 con 0x7f5568103430 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41114/0 (socket says 172.21.15.133:41114) 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556de00640 1 -- 172.21.15.133:0/2631528923 learned_addr learned my addr 172.21.15.133:0/2631528923 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:36512/0 (socket says 172.21.15.133:36512) 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.235+0000 7f556e800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 0x7f55681a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 -- 172.21.15.133:0/2631528923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 msgr2=0x7f55681a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 0x7f55681a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 -- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 msgr2=0x7f5568199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 -- 172.21.15.133:0/2631528923 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681a19f0 con 0x7f5568102a70 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556de00640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f556d400640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9f0 tx=0x7f555800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f5562a00640 1 -- 172.21.15.133:0/2631528923 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800cd80 con 0x7f5568102a70 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f5562a00640 1 -- 172.21.15.133:0/2631528923 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55580035f0 con 0x7f5568102a70 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.236+0000 7f5562a00640 1 -- 172.21.15.133:0/2631528923 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558010690 con 0x7f5568102a70 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.237+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681a1ce0 con 0x7f5568102a70 2024-05-09T08:29:37.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.237+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a2180 con 0x7f5568102a70 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 Processor -- start 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 -- start start 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f5562a00640 1 -- 172.21.15.133:0/2631528923 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5558003b30 con 0x7f5568102a70 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bd5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f086b31480 con 0x55f086bcdfb0 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f086ad95c0 con 0x55f086bce9a0 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f55737a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f086b2d440 con 0x55f086bd3b10 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f556e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f556e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41116/0 (socket says 172.21.15.133:41116) 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f556de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556d400640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bd5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.240+0000 7f556e800640 1 -- 172.21.15.133:0/452861168 learned_addr learned my addr 172.21.15.133:0/452861168 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 -- 172.21.15.133:0/452861168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 msgr2=0x55f086bd5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bd5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 -- 172.21.15.133:0/452861168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 msgr2=0x55f086bce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 -- 172.21.15.133:0/452861168 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0868452d0 con 0x55f086bce9a0 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556e800640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556d400640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bd5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.241+0000 7f556de00640 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd34a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f555c005ab0 tx=0x7f555c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b9c35694a90bb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f5557000640 1 -- 172.21.15.133:0/452861168 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c035040 con 0x55f086bce9a0 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f5557000640 1 -- 172.21.15.133:0/452861168 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c0319b0 con 0x55f086bce9a0 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f5562a00640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 0x7f55500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f5557000640 1 -- 172.21.15.133:0/452861168 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c004a00 con 0x55f086bce9a0 2024-05-09T08:29:37.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f5562a00640 1 -- 172.21.15.133:0/2631528923 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f55580cf370 con 0x7f5568102a70 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f556e800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 0x7f55500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 -- 172.21.15.133:0/452861168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 msgr2=0x55f086bd34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd34a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f555c005ab0 tx=0x7f555c005de0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 -- 172.21.15.133:0/452861168 shutdown_connections 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bd5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd34a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 --2- 172.21.15.133:0/452861168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.242+0000 7f55737a1740 1 -- 172.21.15.133:0/452861168 >> 172.21.15.133:0/452861168 conn(0x55f086b958a0 msgr2=0x55f086bc9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.243+0000 7f55737a1740 1 -- 172.21.15.133:0/452861168 shutdown_connections 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.243+0000 7f556e800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 0x7f55500a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55640099a0 tx=0x7f5564009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.243+0000 7f55737a1740 1 -- 172.21.15.133:0/452861168 wait complete. 2024-05-09T08:29:37.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.243+0000 7f55737a1740 1 -- start start 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.243+0000 7f55737a1740 1 Processor -- start 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bdf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f086bd6a60 con 0x55f086bcdfb0 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f086aec980 con 0x55f086bce9a0 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f086bd6be0 con 0x55f086bd3b10 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f55737a1740 10 client.0 _ll_register_callbacks cb 0x55f086b27fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41124/0 (socket says 172.21.15.133:41124) 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 -- 172.21.15.133:0/2148887569 learned_addr learned my addr 172.21.15.133:0/2148887569 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556d400640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556e800640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bdf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 -- 172.21.15.133:0/2148887569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 msgr2=0x55f086bdf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bdf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 -- 172.21.15.133:0/2148887569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 msgr2=0x55f086bd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556de00640 1 -- 172.21.15.133:0/2148887569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f08690f6b0 con 0x55f086bcdfb0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.244+0000 7f556e800640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bdf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f556de00640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f555c031440 tx=0x7f555c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137981e1ce9686f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f554b400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c04e070 con 0x55f086bcdfb0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f555c0035f0 con 0x55f086bcdfb0 2024-05-09T08:29:37.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555c0319b0 con 0x55f086bcdfb0 2024-05-09T08:29:37.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f555c05a050 con 0x55f086bcdfb0 2024-05-09T08:29:37.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f55737a1740 10 client.14874 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.245+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f086be2630 con 0x55f086bcdfb0 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c0319b0 con 0x55f086bcdfb0 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f555c046870 con 0x55f086bcdfb0 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 10 client.14874 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 10 client.14874 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 20 client.14874 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086be4770 0x55f086be5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 4 client.14874 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 --> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f086b45060 con 0x55f086be4770 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.246+0000 7f556d400640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086be4770 0x55f086be5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.247+0000 7f556d400640 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086be4770 0x55f086be5be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=829b6414c523bcf0 server_cookie=837e4e8e24b7b178 in_seq=0 out_seq=0 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.247+0000 7f554b400640 10 client.14874 ms_handle_connect on v2:172.21.15.133:6838/3807326844 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.247+0000 7f554b400640 1 -- 172.21.15.133:0/2148887569 <== mds.2 v2:172.21.15.133:6838/3807326844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f086b45060 con 0x55f086be4770 2024-05-09T08:29:37.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.247+0000 7f554b400640 10 client.14874 handle_command_reply: tid=0 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.247+0000 7f55737a1740 1 client.14874 shutdown 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 msgr2=0x55f086bd80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f555c031440 tx=0x7f555c02a410 comp rx=0 tx=0).stop 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 shutdown_connections 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086be4770 0x55f086be5be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bd3b10 0x55f086bdf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 --2- 172.21.15.133:0/2148887569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bd80d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f554b400640 0 client.14874 ms_handle_reset on v2:172.21.15.133:6838/3807326844 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 >> 172.21.15.133:0/2148887569 conn(0x55f086b958a0 msgr2=0x55f086bc8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 shutdown_connections 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 1 -- 172.21.15.133:0/2148887569 wait complete. 2024-05-09T08:29:37.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.248+0000 7f55737a1740 20 client.14874 trim_cache size 0 max 16384 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 Processor -- start 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 -- start start 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 0x55f086bdf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bd3b10 0x55f086bcf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f086be3f20 con 0x55f086bcdfb0 2024-05-09T08:29:37.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f086be45f0 con 0x55f086bd3b10 2024-05-09T08:29:37.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.268+0000 7f55737a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f086be4cc0 con 0x55f086bce9a0 2024-05-09T08:29:37.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41126/0 (socket says 172.21.15.133:41126) 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556de00640 1 -- 172.21.15.133:0/1387079961 learned_addr learned my addr 172.21.15.133:0/1387079961 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 0x55f086bdf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556e800640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bd3b10 0x55f086bcf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 -- 172.21.15.133:0/1387079961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bd3b10 msgr2=0x55f086bcf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bd3b10 0x55f086bcf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 -- 172.21.15.133:0/1387079961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 msgr2=0x55f086bcd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 -- 172.21.15.133:0/1387079961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f086bca6e0 con 0x55f086bce9a0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556de00640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.269+0000 7f556d400640 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 0x55f086bdf570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f555809b8a0 tx=0x7f55580d2b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed4433c47c73c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f5556600640 1 -- 172.21.15.133:0/1387079961 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555809c2a0 con 0x55f086bce9a0 2024-05-09T08:29:37.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f5556600640 1 -- 172.21.15.133:0/1387079961 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555809bba0 con 0x55f086bce9a0 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f5556600640 1 -- 172.21.15.133:0/1387079961 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558098210 con 0x55f086bce9a0 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 -- 172.21.15.133:0/1387079961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 msgr2=0x55f086bdf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 0x55f086bdf570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f555809b8a0 tx=0x7f55580d2b80 comp rx=0 tx=0).stop 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 -- 172.21.15.133:0/1387079961 shutdown_connections 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bd3b10 0x55f086bcf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bce9a0 0x55f086bdf570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 --2- 172.21.15.133:0/1387079961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bcdfb0 0x55f086bcd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 -- 172.21.15.133:0/1387079961 >> 172.21.15.133:0/1387079961 conn(0x55f086b958a0 msgr2=0x55f086bcc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.270+0000 7f55737a1740 1 -- 172.21.15.133:0/1387079961 shutdown_connections 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 -- 172.21.15.133:0/1387079961 wait complete. 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 -- start start 2024-05-09T08:29:37.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 Processor -- start 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bdb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bd3b10 0x55f086bde360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f086be5a60 con 0x55f086bd3b10 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55f086be5390 con 0x55f086bce9a0 2024-05-09T08:29:37.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f086be6130 con 0x55f086bcdfb0 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.271+0000 7f55737a1740 10 client.0 _ll_register_callbacks cb 0x7f555c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bd3b10 0x55f086bde360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:49052/0 (socket says 172.21.15.133:49052) 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 -- 172.21.15.133:0/2332325733 learned_addr learned my addr 172.21.15.133:0/2332325733 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556d400640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bdb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 -- 172.21.15.133:0/2332325733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 msgr2=0x55f086bdb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bdb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 -- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bd3b10 msgr2=0x55f086bde360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bd3b10 0x55f086bde360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 -- 172.21.15.133:0/2332325733 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f086bde950 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f556de00640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f555c031440 tx=0x7f555c005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5dfb6e24d4430d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.272+0000 7f554b400640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c057070 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555809bba0 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f555c046650 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f555c04e050 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 10 client.24973 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f086c4a780 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555809bba0 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f555c0035f0 con 0x55f086bcdfb0 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 10 client.24973 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 10 client.24973 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 20 client.24973 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086c4f350 0x55f086c4e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 4 client.24973 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.273+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 --> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] -- command(tid 0: {"prefix": "status"}) -- 0x55f086b3ebb0 con 0x55f086c4f350 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f556d400640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086c4f350 0x55f086c4e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f556d400640 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086c4f350 0x55f086c4e060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a95d21e41d4632d2 server_cookie=fa2e21186c48ed57 in_seq=0 out_seq=0 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f554b400640 10 client.24973 ms_handle_connect on v2:172.21.15.133:6838/3807326844 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f554b400640 1 -- 172.21.15.133:0/2332325733 <== mds.2 v2:172.21.15.133:6838/3807326844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f086b3ebb0 con 0x55f086c4f350 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f554b400640 10 client.24973 handle_command_reply: tid=0 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.274+0000 7f55737a1740 1 client.24973 shutdown 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 msgr2=0x55f086bdae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f555c031440 tx=0x7f555c005900 comp rx=0 tx=0).stop 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 shutdown_connections 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x55f086c4f350 0x55f086c4e060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f086bd3b10 0x55f086bde360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f086bce9a0 0x55f086bdb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 --2- 172.21.15.133:0/2332325733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f086bcdfb0 0x55f086bdae00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f554b400640 0 client.24973 ms_handle_reset on v2:172.21.15.133:6838/3807326844 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 >> 172.21.15.133:0/2332325733 conn(0x55f086b958a0 msgr2=0x55f086bc82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 shutdown_connections 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 1 -- 172.21.15.133:0/2332325733 wait complete. 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f55737a1740 20 client.24973 trim_cache size 0 max 16384 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24536, 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:29:37.280 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 97.018814316000004, 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 115.228093122 2024-05-09T08:29:37.281 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 msgr2=0x7f55500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 0x7f55500a50d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55640099a0 tx=0x7f5564009500 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 msgr2=0x7f556819a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.275+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9f0 tx=0x7f555800eeb0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 shutdown_connections 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f55500a2c10 0x7f55500a50d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568103430 0x7f55681a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568102a70 0x7f556819a360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 --2- 172.21.15.133:0/2631528923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568101880 0x7f5568199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 >> 172.21.15.133:0/2631528923 conn(0x7f55680fce10 msgr2=0x7f5568108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 shutdown_connections 2024-05-09T08:29:37.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.276+0000 7f5570800640 1 -- 172.21.15.133:0/2631528923 wait complete. 2024-05-09T08:29:37.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T08:29:37.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.485+0000 7f2758c00640 1 -- 172.21.15.133:0/1139686689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 msgr2=0x7f2754103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 --2- 172.21.15.133:0/1139686689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754103d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2744009900 tx=0x7f274402a410 comp rx=0 tx=0).stop 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 -- 172.21.15.133:0/1139686689 shutdown_connections 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 --2- 172.21.15.133:0/1139686689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754105520 0x7f275410bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 --2- 172.21.15.133:0/1139686689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754104b60 0x7f2754104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 --2- 172.21.15.133:0/1139686689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754103d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 -- 172.21.15.133:0/1139686689 >> 172.21.15.133:0/1139686689 conn(0x7f27540feee0 msgr2=0x7f2754101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 -- 172.21.15.133:0/1139686689 shutdown_connections 2024-05-09T08:29:37.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.486+0000 7f2758c00640 1 -- 172.21.15.133:0/1139686689 wait complete. 2024-05-09T08:29:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 Processor -- start 2024-05-09T08:29:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 -- start start 2024-05-09T08:29:37.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754105520 0x7f27541a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f275410f610 con 0x7f2754103970 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f275410f490 con 0x7f2754104b60 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2758c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f275410f790 con 0x7f2754105520 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f274aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f2753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.487+0000 7f274aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:36564/0 (socket says 172.21.15.133:36564) 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 -- 172.21.15.133:0/2716053826 learned_addr learned my addr 172.21.15.133:0/2716053826 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f2753e00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754105520 0x7f27541a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 -- 172.21.15.133:0/2716053826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754105520 msgr2=0x7f27541a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754105520 0x7f27541a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 -- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 msgr2=0x7f2754199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 -- 172.21.15.133:0/2716053826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541a18f0 con 0x7f2754104b60 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f274aa00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f273c00ea60 tx=0x7f273c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f2750c00640 1 -- 172.21.15.133:0/2716053826 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c00cde0 con 0x7f2754104b60 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f2750c00640 1 -- 172.21.15.133:0/2716053826 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f273c0035f0 con 0x7f2754104b60 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.488+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27541a1be0 con 0x7f2754104b60 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.489+0000 7f2750c00640 1 -- 172.21.15.133:0/2716053826 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c010670 con 0x7f2754104b60 2024-05-09T08:29:37.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.489+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a2100 con 0x7f2754104b60 2024-05-09T08:29:37.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.491+0000 7f275d1c1740 1 Processor -- start 2024-05-09T08:29:37.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 -- start start 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f2750c00640 1 -- 172.21.15.133:0/2716053826 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f273c0037a0 con 0x7f2754104b60 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17b8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd17bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555cd1716480 con 0x555cd17b3190 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555cd16be5c0 con 0x555cd17b8cf0 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f275d1c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cd1712440 con 0x555cd17b3b80 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f2753e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f2753e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41164/0 (socket says 172.21.15.133:41164) 2024-05-09T08:29:37.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f2753e00640 1 -- 172.21.15.133:0/434196648 learned_addr learned my addr 172.21.15.133:0/434196648 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f274aa00640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd17bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.492+0000 7f2753400640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 -- 172.21.15.133:0/434196648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 msgr2=0x555cd17bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd17bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 -- 172.21.15.133:0/434196648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 msgr2=0x555cd17b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 -- 172.21.15.133:0/434196648 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd142a2d0 con 0x555cd17b3b80 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753e00640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2753400640 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17b8680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2744009230 tx=0x7f27440040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76c40faca8b3d114 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.493+0000 7f2743e00640 1 -- 172.21.15.133:0/434196648 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035040 con 0x555cd17b3b80 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f2743e00640 1 -- 172.21.15.133:0/434196648 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440044b0 con 0x555cd17b3b80 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f2743e00640 1 -- 172.21.15.133:0/434196648 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440316b0 con 0x555cd17b3b80 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f2750c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 0x7f27380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f2750c00640 1 -- 172.21.15.133:0/2716053826 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f273c0cf320 con 0x7f2754104b60 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f2753e00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 0x7f27380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 -- 172.21.15.133:0/434196648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 msgr2=0x555cd17b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17b8680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2744009230 tx=0x7f27440040c0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 -- 172.21.15.133:0/434196648 shutdown_connections 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd17bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17b8680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 --2- 172.21.15.133:0/434196648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.494+0000 7f275d1c1740 1 -- 172.21.15.133:0/434196648 >> 172.21.15.133:0/434196648 conn(0x555cd177a8a0 msgr2=0x555cd17aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.495+0000 7f275d1c1740 1 -- 172.21.15.133:0/434196648 shutdown_connections 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.495+0000 7f2753e00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 0x7f27380a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x555cd17b43e0 tx=0x7f274c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.495+0000 7f275d1c1740 1 -- 172.21.15.133:0/434196648 wait complete. 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.495+0000 7f275d1c1740 1 -- start start 2024-05-09T08:29:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.495+0000 7f275d1c1740 1 Processor -- start 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b8cf0 0x555cd17c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555cd17bbc40 con 0x555cd17b8cf0 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555cd16d1980 con 0x555cd17b3b80 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cd17bbdc0 con 0x555cd17b3190 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f275d1c1740 10 client.0 _ll_register_callbacks cb 0x555cd170cfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:49078/0 (socket says 172.21.15.133:49078) 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f274aa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b8cf0 0x555cd17c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 -- 172.21.15.133:0/970621059 learned_addr learned my addr 172.21.15.133:0/970621059 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 -- 172.21.15.133:0/970621059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 msgr2=0x555cd17bd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17bd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 -- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b8cf0 msgr2=0x555cd17c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b8cf0 0x555cd17c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f2753400640 1 -- 172.21.15.133:0/970621059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd14f46b0 con 0x555cd17b3190 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.496+0000 7f274aa00640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2753400640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f274402aa00 tx=0x7f2744009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a5f3b7abb10d21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2740c00640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035070 con 0x555cd17b3190 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f27440318c0 con 0x555cd17b3190 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27440044b0 con 0x555cd17b3190 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2744031ac0 con 0x555cd17b3190 2024-05-09T08:29:37.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f275d1c1740 10 client.24985 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555cd17bbc40 con 0x555cd17b3190 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.497+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27440044b0 con 0x555cd17b3190 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f2744004ce0 con 0x555cd17b3190 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 10 client.24985 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 10 client.24985 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 20 client.24985 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd17c9910 0x555cd17cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 4 client.24985 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 --> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd172a060 con 0x555cd17c9910 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f274aa00640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd17c9910 0x555cd17cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f274aa00640 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd17c9910 0x555cd17cad80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86e5a1f480434417 server_cookie=7bc5da6b05f7ab22 in_seq=0 out_seq=0 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.498+0000 7f2740c00640 10 client.24985 ms_handle_connect on v2:172.21.15.133:6840/2374052560 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.499+0000 7f2740c00640 1 -- 172.21.15.133:0/970621059 <== mds.1 v2:172.21.15.133:6840/2374052560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cd172a060 con 0x555cd17c9910 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.499+0000 7f2740c00640 10 client.24985 handle_command_reply: tid=0 2024-05-09T08:29:37.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.499+0000 7f275d1c1740 1 client.24985 shutdown 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 msgr2=0x555cd17bd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f274402aa00 tx=0x7f2744009260 comp rx=0 tx=0).stop 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 shutdown_connections 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd17c9910 0x555cd17cad80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b8cf0 0x555cd17c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 --2- 172.21.15.133:0/970621059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3190 0x555cd17bd2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f2740c00640 0 client.24985 ms_handle_reset on v2:172.21.15.133:6840/2374052560 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 >> 172.21.15.133:0/970621059 conn(0x555cd177a8a0 msgr2=0x555cd17adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 shutdown_connections 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 1 -- 172.21.15.133:0/970621059 wait complete. 2024-05-09T08:29:37.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.500+0000 7f275d1c1740 20 client.24985 trim_cache size 0 max 16384 2024-05-09T08:29:37.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.519+0000 7f275d1c1740 1 Processor -- start 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.519+0000 7f275d1c1740 1 -- start start 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 0x555cd17b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555cd17c90c0 con 0x555cd17b3190 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555cd17c9790 con 0x555cd17b3b80 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f275d1c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cd17c9e60 con 0x555cd17b8cf0 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41192/0 (socket says 172.21.15.133:41192) 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 -- 172.21.15.133:0/3084870671 learned_addr learned my addr 172.21.15.133:0/3084870671 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753e00640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 0x555cd17b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f274aa00640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 -- 172.21.15.133:0/3084870671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 msgr2=0x555cd17b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 0x555cd17b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 -- 172.21.15.133:0/3084870671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 msgr2=0x555cd17c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753400640 1 -- 172.21.15.133:0/3084870671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd17aef40 con 0x555cd17b3190 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f2753e00640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 0x555cd17b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.520+0000 7f274aa00640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f2753400640 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2744006010 tx=0x7f27440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3878b54517a7722b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f2743400640 1 -- 172.21.15.133:0/3084870671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744035070 con 0x555cd17b3190 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f2743400640 1 -- 172.21.15.133:0/3084870671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274404a070 con 0x555cd17b3190 2024-05-09T08:29:37.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f2743400640 1 -- 172.21.15.133:0/3084870671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440038e0 con 0x555cd17b3190 2024-05-09T08:29:37.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f275d1c1740 1 -- 172.21.15.133:0/3084870671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 msgr2=0x555cd17b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f275d1c1740 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2744006010 tx=0x7f27440040c0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.521+0000 7f275d1c1740 1 -- 172.21.15.133:0/3084870671 shutdown_connections 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b8cf0 0x555cd17b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b3b80 0x555cd17c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 --2- 172.21.15.133:0/3084870671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17b2470 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 -- 172.21.15.133:0/3084870671 >> 172.21.15.133:0/3084870671 conn(0x555cd177a8a0 msgr2=0x555cd17a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 -- 172.21.15.133:0/3084870671 shutdown_connections 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 -- 172.21.15.133:0/3084870671 wait complete. 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 -- start start 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.522+0000 7f275d1c1740 1 Processor -- start 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17bfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd182d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555cd17cac00 con 0x555cd17b3190 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x555cd17ca530 con 0x555cd17b8cf0 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cd17cb2d0 con 0x555cd17b3b80 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f275d1c1740 10 client.0 _ll_register_callbacks cb 0x7f274c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41208/0 (socket says 172.21.15.133:41208) 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 -- 172.21.15.133:0/4080985304 learned_addr learned my addr 172.21.15.133:0/4080985304 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753e00640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd182d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f274aa00640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17bfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 -- 172.21.15.133:0/4080985304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 msgr2=0x555cd17bfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17bfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 -- 172.21.15.133:0/4080985304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 msgr2=0x555cd182d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd182d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 -- 172.21.15.133:0/4080985304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd17c4a40 con 0x555cd17b3190 2024-05-09T08:29:37.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2753400640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2744005130 tx=0x7f27440315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caca880a425248b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.523+0000 7f2740c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744067060 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2744004e30 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274404a070 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2744035070 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f275d1c1740 10 client.14898 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555cd182eda0 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274404a070 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f274406e780 con 0x555cd17b3190 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.524+0000 7f275d1c1740 10 client.14898 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f275d1c1740 10 client.14898 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f275d1c1740 20 client.14898 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd1833570 0x555cd1832280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f275d1c1740 4 client.14898 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 --> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] -- command(tid 0: {"prefix": "status"}) -- 0x555cd17b8a60 con 0x555cd1833570 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f274aa00640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd1833570 0x555cd1832280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f2740c00640 10 client.14898 ms_handle_connect on v2:172.21.15.133:6840/2374052560 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.525+0000 7f274aa00640 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd1833570 0x555cd1832280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8f416375df0a40b server_cookie=6b4438a2eab57e64 in_seq=0 out_seq=0 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f2740c00640 1 -- 172.21.15.133:0/4080985304 <== mds.1 v2:172.21.15.133:6840/2374052560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555cd17b8a60 con 0x555cd1833570 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f2740c00640 10 client.14898 handle_command_reply: tid=0 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 client.14898 shutdown 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 msgr2=0x555cd17bf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2744005130 tx=0x7f27440315b0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 shutdown_connections 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x555cd1833570 0x555cd1832280 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cd17b8cf0 0x555cd182d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f2740c00640 0 client.14898 ms_handle_reset on v2:172.21.15.133:6840/2374052560 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cd17b3b80 0x555cd17bfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 --2- 172.21.15.133:0/4080985304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555cd17b3190 0x555cd17bf510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 >> 172.21.15.133:0/4080985304 conn(0x555cd177a8a0 msgr2=0x555cd17b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 shutdown_connections 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 1 -- 172.21.15.133:0/4080985304 wait complete. 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.526+0000 7f275d1c1740 20 client.14898 trim_cache size 0 max 16384 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24556, 2024-05-09T08:29:37.531 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 98.279830071000006, 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 111.368277263 2024-05-09T08:29:37.532 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 msgr2=0x7f27380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 0x7f27380a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x555cd17b43e0 tx=0x7f274c009500 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 msgr2=0x7f275419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f273c00ea60 tx=0x7f273c00ef20 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 shutdown_connections 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f27380a2b90 0x7f27380a5050 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2754105520 0x7f27541a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2754104b60 0x7f275419a1f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 --2- 172.21.15.133:0/2716053826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2754103970 0x7f2754199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 >> 172.21.15.133:0/2716053826 conn(0x7f27540feee0 msgr2=0x7f2754107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 shutdown_connections 2024-05-09T08:29:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.527+0000 7f2758c00640 1 -- 172.21.15.133:0/2716053826 wait complete. 2024-05-09T08:29:37.545 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 -- 172.21.15.133:0/3686054111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7381029b0 msgr2=0x7fc738102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 --2- 172.21.15.133:0/3686054111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7381029b0 0x7fc738102e30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc720009900 tx=0x7fc72002a410 comp rx=0 tx=0).stop 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 -- 172.21.15.133:0/3686054111 shutdown_connections 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 --2- 172.21.15.133:0/3686054111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc738103370 0x7fc738109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 --2- 172.21.15.133:0/3686054111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7381029b0 0x7fc738102e30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 --2- 172.21.15.133:0/3686054111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 0x7fc738101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 -- 172.21.15.133:0/3686054111 >> 172.21.15.133:0/3686054111 conn(0x7fc7380fcd30 msgr2=0x7fc7380ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 -- 172.21.15.133:0/3686054111 shutdown_connections 2024-05-09T08:29:37.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.741+0000 7fc73d400640 1 -- 172.21.15.133:0/3686054111 wait complete. 2024-05-09T08:29:37.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 Processor -- start 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 -- start start 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 0x7fc738199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7381029b0 0x7fc73819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc73810d420 con 0x7fc738103370 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc73810d2a0 con 0x7fc7381029b0 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.742+0000 7fc73d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc73810d5a0 con 0x7fc7381017c0 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41224/0 (socket says 172.21.15.133:41224) 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 -- 172.21.15.133:0/173509424 learned_addr learned my addr 172.21.15.133:0/173509424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc737400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7381029b0 0x7fc73819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc737e00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 0x7fc738199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 -- 172.21.15.133:0/173509424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 msgr2=0x7fc738199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 0x7fc738199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 -- 172.21.15.133:0/173509424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7381029b0 msgr2=0x7fc73819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7381029b0 0x7fc73819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 -- 172.21.15.133:0/173509424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7381a18e0 con 0x7fc738103370 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.743+0000 7fc73ca00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e970 tx=0x7fc73000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.744+0000 7fc734c00640 1 -- 172.21.15.133:0/173509424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73000ccc0 con 0x7fc738103370 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.744+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7381a1bd0 con 0x7fc738103370 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.744+0000 7fc734c00640 1 -- 172.21.15.133:0/173509424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73000ce60 con 0x7fc738103370 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.744+0000 7fc734c00640 1 -- 172.21.15.133:0/173509424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730010490 con 0x7fc738103370 2024-05-09T08:29:37.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.744+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7381a20f0 con 0x7fc738103370 2024-05-09T08:29:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc734c00640 1 -- 172.21.15.133:0/173509424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc730005020 con 0x7fc738103370 2024-05-09T08:29:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 Processor -- start 2024-05-09T08:29:37.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 -- start start 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5b80 0x5630412ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412bd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563041218480 con 0x5630412b5190 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5630411c05c0 con 0x5630412b5b80 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.747+0000 7fc741835740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563041214440 con 0x5630412bacf0 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41234/0 (socket says 172.21.15.133:41234) 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 -- 172.21.15.133:0/2994426265 learned_addr learned my addr 172.21.15.133:0/2994426265 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737e00640 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5b80 0x5630412ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 -- 172.21.15.133:0/2994426265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 msgr2=0x5630412bd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412bd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 -- 172.21.15.133:0/2994426265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5b80 msgr2=0x5630412ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5b80 0x5630412ba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 -- 172.21.15.133:0/2994426265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563040f2c2d0 con 0x5630412b5190 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.748+0000 7fc737400640 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc720006010 tx=0x7fc720005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8222436ca66a53e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc734c00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 0x7fc71c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc72d200640 1 -- 172.21.15.133:0/2994426265 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720044020 con 0x5630412b5190 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc734c00640 1 -- 172.21.15.133:0/173509424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc7300cf460 con 0x7fc738103370 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc737e00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 0x7fc71c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc72d200640 1 -- 172.21.15.133:0/2994426265 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720004770 con 0x5630412b5190 2024-05-09T08:29:37.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.749+0000 7fc72d200640 1 -- 172.21.15.133:0/2994426265 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720035040 con 0x5630412b5190 2024-05-09T08:29:37.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- 172.21.15.133:0/2994426265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 msgr2=0x5630412b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc720006010 tx=0x7fc720005e50 comp rx=0 tx=0).stop 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc737e00640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 0x7fc71c0a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5630412b5f80 tx=0x7fc728009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- 172.21.15.133:0/2994426265 shutdown_connections 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412bd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5b80 0x5630412ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 --2- 172.21.15.133:0/2994426265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5190 0x5630412b55b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- 172.21.15.133:0/2994426265 >> 172.21.15.133:0/2994426265 conn(0x56304127c8a0 msgr2=0x5630412b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- 172.21.15.133:0/2994426265 shutdown_connections 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- 172.21.15.133:0/2994426265 wait complete. 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.750+0000 7fc741835740 1 -- start start 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 Processor -- start 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5630412bdc40 con 0x5630412b5b80 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5630411d3980 con 0x5630412b5190 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5630412bddc0 con 0x5630412bacf0 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc741835740 10 client.0 _ll_register_callbacks cb 0x56304120efc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc737400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc737e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc737400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41248/0 (socket says 172.21.15.133:41248) 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.751+0000 7fc737400640 1 -- 172.21.15.133:0/3098682346 learned_addr learned my addr 172.21.15.133:0/3098682346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc73ca00640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 -- 172.21.15.133:0/3098682346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 msgr2=0x5630412c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 -- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 msgr2=0x5630412bf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 -- 172.21.15.133:0/3098682346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563040ff66b0 con 0x5630412b5b80 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737e00640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc73ca00640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc737400640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc72002a450 tx=0x7fc720004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abea3a568dab414 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.752+0000 7fc725600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72004d020 con 0x5630412b5b80 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc720005200 con 0x5630412b5b80 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc720004770 con 0x5630412b5b80 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc720035070 con 0x5630412b5b80 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc741835740 10 client.14910 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5630412c9800 con 0x5630412b5b80 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720004770 con 0x5630412b5b80 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc720003680 con 0x5630412b5b80 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.753+0000 7fc741835740 10 client.14910 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc741835740 10 client.14910 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc741835740 20 client.14910 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x5630412cab50 0x5630412ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc741835740 4 client.14910 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 --> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56304122c060 con 0x5630412cab50 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.754+0000 7fc737e00640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x5630412cab50 0x5630412ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.755+0000 7fc737e00640 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x5630412cab50 0x5630412ccd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2458f8f1091bf13f server_cookie=483b1b201ff887f0 in_seq=0 out_seq=0 2024-05-09T08:29:37.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.755+0000 7fc725600640 10 client.14910 ms_handle_connect on v2:172.21.15.160:6838/145157679 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.755+0000 7fc725600640 1 -- 172.21.15.133:0/3098682346 <== mds.0 v2:172.21.15.160:6838/145157679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56304122c060 con 0x5630412cab50 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.755+0000 7fc725600640 10 client.14910 handle_command_reply: tid=0 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.755+0000 7fc741835740 1 client.14910 shutdown 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 msgr2=0x5630412bf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc72002a450 tx=0x7fc720004b50 comp rx=0 tx=0).stop 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 shutdown_connections 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412bf850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x5630412cab50 0x5630412ccd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 --2- 172.21.15.133:0/3098682346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 >> 172.21.15.133:0/3098682346 conn(0x56304127c8a0 msgr2=0x5630412afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc725600640 0 client.14910 ms_handle_reset on v2:172.21.15.160:6838/145157679 2024-05-09T08:29:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 shutdown_connections 2024-05-09T08:29:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 1 -- 172.21.15.133:0/3098682346 wait complete. 2024-05-09T08:29:37.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.756+0000 7fc741835740 20 client.14910 trim_cache size 0 max 16384 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 Processor -- start 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 -- start start 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 0x5630412c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 0x5630412bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5630412cb090 con 0x5630412bacf0 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5630412cb760 con 0x5630412b5190 2024-05-09T08:29:37.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc741835740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5630412cbe30 con 0x5630412b5b80 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc73ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 0x5630412bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc73ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 0x5630412bd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41262/0 (socket says 172.21.15.133:41262) 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc73ca00640 1 -- 172.21.15.133:0/204680097 learned_addr learned my addr 172.21.15.133:0/204680097 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc737400640 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 0x5630412c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.776+0000 7fc737e00640 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 -- 172.21.15.133:0/204680097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 msgr2=0x5630412bdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 -- 172.21.15.133:0/204680097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 msgr2=0x5630412bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 0x5630412bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 -- 172.21.15.133:0/204680097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630412b63a0 con 0x5630412b5b80 2024-05-09T08:29:37.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc737400640 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 0x5630412c67e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc720031440 tx=0x7fc720005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c99c5669c76c5649 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc727e00640 1 -- 172.21.15.133:0/204680097 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72004d070 con 0x5630412b5b80 2024-05-09T08:29:37.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc727e00640 1 -- 172.21.15.133:0/204680097 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720005850 con 0x5630412b5b80 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.777+0000 7fc727e00640 1 -- 172.21.15.133:0/204680097 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720044040 con 0x5630412b5b80 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- 172.21.15.133:0/204680097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 msgr2=0x5630412c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 0x5630412c67e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc720031440 tx=0x7fc720005e30 comp rx=0 tx=0).stop 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- 172.21.15.133:0/204680097 shutdown_connections 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412bacf0 0x5630412bd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412b5b80 0x5630412c67e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 --2- 172.21.15.133:0/204680097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412bdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- 172.21.15.133:0/204680097 >> 172.21.15.133:0/204680097 conn(0x56304127c8a0 msgr2=0x5630412ba330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- 172.21.15.133:0/204680097 shutdown_connections 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- 172.21.15.133:0/204680097 wait complete. 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.778+0000 7fc741835740 1 -- start start 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 Processor -- start 2024-05-09T08:29:37.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5630412ccbd0 con 0x5630412b5b80 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5630412cc500 con 0x5630412b5190 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5630412cd2a0 con 0x5630412bacf0 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc741835740 10 client.0 _ll_register_callbacks cb 0x7fc728009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc737e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc737400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc737e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:36652/0 (socket says 172.21.15.133:36652) 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc737e00640 1 -- 172.21.15.133:0/1900502279 learned_addr learned my addr 172.21.15.133:0/1900502279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.779+0000 7fc73ca00640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 -- 172.21.15.133:0/1900502279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 msgr2=0x5630412c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 -- 172.21.15.133:0/1900502279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 msgr2=0x5630412c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:29:37.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 -- 172.21.15.133:0/1900502279 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563041338090 con 0x5630412b5190 2024-05-09T08:29:37.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737400640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc73ca00640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:37.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc737e00640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc73819ab70 tx=0x7fc728004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b646aecadbe3b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:37.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc725600640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728004b20 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc728004cc0 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.780+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc720005850 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc72802a380 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 10 client.25016 fetch_fsmap learned FSMap version 22 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563041339370 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc720005850 con 0x5630412b5190 2024-05-09T08:29:37.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fc728005790 con 0x5630412b5190 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 10 client.25016 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 10 client.25016 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 20 client.25016 populate_metadata read hostname 'smithi133' 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x56304133e000 0x56304133cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 4 client.25016 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.781+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 --> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] -- command(tid 0: {"prefix": "status"}) -- 0x563041225bb0 con 0x56304133e000 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.782+0000 7fc737400640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x56304133e000 0x56304133cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.782+0000 7fc737400640 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x56304133e000 0x56304133cd10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ddefa1d1c938e9 server_cookie=8e6689d8222fbe0c in_seq=0 out_seq=0 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.782+0000 7fc725600640 10 client.25016 ms_handle_connect on v2:172.21.15.160:6838/145157679 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc725600640 1 -- 172.21.15.133:0/1900502279 <== mds.0 v2:172.21.15.160:6838/145157679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563041225bb0 con 0x56304133e000 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc725600640 10 client.25016 handle_command_reply: tid=0 2024-05-09T08:29:37.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 client.25016 shutdown 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 msgr2=0x5630412c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc73819ab70 tx=0x7fc728004a40 comp rx=0 tx=0).stop 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 shutdown_connections 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630412bacf0 0x5630412c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5630412b5b80 0x5630412c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x56304133e000 0x56304133cd10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 --2- 172.21.15.133:0/1900502279 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5630412b5190 0x5630412c1130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc725600640 0 client.25016 ms_handle_reset on v2:172.21.15.160:6838/145157679 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 >> 172.21.15.133:0/1900502279 conn(0x56304127c8a0 msgr2=0x5630412b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 shutdown_connections 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 1 -- 172.21.15.133:0/1900502279 wait complete. 2024-05-09T08:29:37.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.783+0000 7fc741835740 20 client.25016 trim_cache size 0 max 16384 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24566, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 101.564943883, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 108.757464709 2024-05-09T08:29:37.787 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:29:37.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 msgr2=0x7fc71c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 0x7fc71c0a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5630412b5f80 tx=0x7fc728009500 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 msgr2=0x7fc7381a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e970 tx=0x7fc73000ed70 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 shutdown_connections 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fc71c0a2c10 0x7fc71c0a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc738103370 0x7fc7381a1240 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7381029b0 0x7fc73819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 --2- 172.21.15.133:0/173509424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7381017c0 0x7fc738199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 >> 172.21.15.133:0/173509424 conn(0x7fc7380fcd30 msgr2=0x7fc738073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 shutdown_connections 2024-05-09T08:29:37.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:37.784+0000 7fc73d400640 1 -- 172.21.15.133:0/173509424 wait complete. 2024-05-09T08:29:37.802 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T08:29:37.802 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T08:29:37.812 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T08:29:37.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T08:29:37.875 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:37 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/845508951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:37 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/845508951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:37 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/845508951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:29:38.232 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:38 smithi160 ceph-mon[34070]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T08:29:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:38 smithi133 ceph-mon[29610]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T08:29:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:38 smithi188 ceph-mon[34645]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 -- 172.21.15.133:0/4112090096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 msgr2=0x7f3c901027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 --2- 172.21.15.133:0/4112090096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901027b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c740099c0 tx=0x7f3c74009cd0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 -- 172.21.15.133:0/4112090096 shutdown_connections 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 --2- 172.21.15.133:0/4112090096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103f30 0x7f3c9010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 --2- 172.21.15.133:0/4112090096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901027b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 --2- 172.21.15.133:0/4112090096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103590 0x7f3c901039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 -- 172.21.15.133:0/4112090096 >> 172.21.15.133:0/4112090096 conn(0x7f3c900fd8f0 msgr2=0x7f3c900ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 -- 172.21.15.133:0/4112090096 shutdown_connections 2024-05-09T08:29:40.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.528+0000 7f3c97a00700 1 -- 172.21.15.133:0/4112090096 wait complete. 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 Processor -- start 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 -- start start 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 0x7f3c90198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103f30 0x7f3c9019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9010df70 con 0x7f3c90102390 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9010ddf0 con 0x7f3c90103590 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.530+0000 7f3c97a00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9010e0f0 con 0x7f3c90103f30 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c95000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c95000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41284/0 (socket says 172.21.15.133:41284) 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c95000700 1 -- 172.21.15.133:0/1178979877 learned_addr learned my addr 172.21.15.133:0/1178979877 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 0x7f3c90198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:40.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c95a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103f30 0x7f3c9019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 -- 172.21.15.133:0/1178979877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103f30 msgr2=0x7f3c9019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103f30 0x7f3c9019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 -- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 msgr2=0x7f3c901985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.531+0000 7f3c87e00700 1 -- 172.21.15.133:0/1178979877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c901a0200 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c87e00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 0x7f3c90198b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c004220 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c7c004d10 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c005020 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c901a04f0 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.532+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c901a09e0 con 0x7f3c90103590 2024-05-09T08:29:40.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.533+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c90069010 con 0x7f3c90103590 2024-05-09T08:29:40.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.535+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3c7c004750 con 0x7f3c90103590 2024-05-09T08:29:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.536+0000 7f3c8e000700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 0x7f3c6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.537+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3c7c0ceff0 con 0x7f3c90103590 2024-05-09T08:29:40.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.537+0000 7f3c95000700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 0x7f3c6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:40.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.537+0000 7f3c95000700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 0x7f3c6c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:40.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.539+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3c7c09bfd0 con 0x7f3c90103590 2024-05-09T08:29:40.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.783+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3c90062760 con 0x7f3c6c0a2c20 2024-05-09T08:29:40.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.785+0000 7f3c8e000700 1 -- 172.21.15.133:0/1178979877 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3c90062760 con 0x7f3c6c0a2c20 2024-05-09T08:29:40.785 INFO:teuthology.orchestra.run.smithi133.stdout:Backend: cephadm 2024-05-09T08:29:40.785 INFO:teuthology.orchestra.run.smithi133.stdout:Available: Yes 2024-05-09T08:29:40.785 INFO:teuthology.orchestra.run.smithi133.stdout:Paused: No 2024-05-09T08:29:40.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 msgr2=0x7f3c6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:40.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 0x7f3c6c0a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3c74000c00 tx=0x7f3c74005fb0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 msgr2=0x7f3c90198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 0x7f3c90198b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00ed00 comp rx=0 tx=0).stop 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 shutdown_connections 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f3c6c0a2c20 0x7f3c6c0a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c90103f30 0x7f3c9019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c90102390 0x7f3c901985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 --2- 172.21.15.133:0/1178979877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c90103590 0x7f3c90198b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 >> 172.21.15.133:0/1178979877 conn(0x7f3c900fd8f0 msgr2=0x7f3c90106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.788+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 shutdown_connections 2024-05-09T08:29:40.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:40.789+0000 7f3c97a00700 1 -- 172.21.15.133:0/1178979877 wait complete. 2024-05-09T08:29:41.162 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:40 smithi133 ceph-mon[29610]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:40 smithi160 ceph-mon[34070]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:40 smithi188 ceph-mon[34645]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T08:29:41.669 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T08:29:41.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:41 smithi133 ceph-mon[29610]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:42.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:41 smithi160 ceph-mon[34070]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:42.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:41 smithi188 ceph-mon[34645]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:42 smithi160 ceph-mon[34070]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:29:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:42 smithi133 ceph-mon[29610]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:29:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:42 smithi188 ceph-mon[34645]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:29:44.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 -- 172.21.15.133:0/700832821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c8106160 msgr2=0x7f29c810ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:44.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 --2- 172.21.15.133:0/700832821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c8106160 0x7f29c810ca10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f29c000b0d0 tx=0x7f29c000b3e0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 -- 172.21.15.133:0/700832821 shutdown_connections 2024-05-09T08:29:44.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 --2- 172.21.15.133:0/700832821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c8106160 0x7f29c810ca10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 --2- 172.21.15.133:0/700832821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c81045c0 0x7f29c81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 --2- 172.21.15.133:0/700832821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81057c0 0x7f29c8105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 -- 172.21.15.133:0/700832821 >> 172.21.15.133:0/700832821 conn(0x7f29c80ffb80 msgr2=0x7f29c8101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:44.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 -- 172.21.15.133:0/700832821 shutdown_connections 2024-05-09T08:29:44.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.226+0000 7f29d0000700 1 -- 172.21.15.133:0/700832821 wait complete. 2024-05-09T08:29:44.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.227+0000 7f29d0000700 1 Processor -- start 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.227+0000 7f29d0000700 1 -- start start 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c81057c0 0x7f29c819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c8106160 0x7f29c81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29c81101f0 con 0x7f29c81057c0 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8110070 con 0x7f29c81045c0 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29d0000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8110370 con 0x7f29c8106160 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29ccc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c81057c0 0x7f29c819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:50324/0 (socket says 172.21.15.133:50324) 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 -- 172.21.15.133:0/2158969264 learned_addr learned my addr 172.21.15.133:0/2158969264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29ce000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c8106160 0x7f29c81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:44.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 -- 172.21.15.133:0/2158969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c8106160 msgr2=0x7f29c81a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c8106160 0x7f29c81a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 -- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c81057c0 msgr2=0x7f29c819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c81057c0 0x7f29c819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.228+0000 7f29cd600700 1 -- 172.21.15.133:0/2158969264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c81a2380 con 0x7f29c81045c0 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29cd600700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e950 tx=0x7f29bc00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc004230 con 0x7f29c81045c0 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c81a2670 con 0x7f29c81045c0 2024-05-09T08:29:44.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29bc0043d0 con 0x7f29c81045c0 2024-05-09T08:29:44.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc003dc0 con 0x7f29c81045c0 2024-05-09T08:29:44.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.229+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81a2b60 con 0x7f29c81045c0 2024-05-09T08:29:44.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.230+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c8069010 con 0x7f29c81045c0 2024-05-09T08:29:44.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.232+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f29bc044b30 con 0x7f29c81045c0 2024-05-09T08:29:44.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.233+0000 7f29c6000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 0x7f29b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:44.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.234+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f29bc0cf270 con 0x7f29c81045c0 2024-05-09T08:29:44.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.234+0000 7f29ccc00700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 0x7f29b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:44.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.234+0000 7f29ccc00700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 0x7f29b00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009680 tx=0x7f29b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:44.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.236+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f29bc044e20 con 0x7f29c81045c0 2024-05-09T08:29:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.472+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f29c8062760 con 0x7f29b00a2c20 2024-05-09T08:29:44.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.488+0000 7f29c6000700 1 -- 172.21.15.133:0/2158969264 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f29c8062760 con 0x7f29b00a2c20 2024-05-09T08:29:44.488 INFO:teuthology.orchestra.run.smithi133.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.a smithi133 running (118s) 23s ago 118s 16.5M - 19.0.0-2515-g96805dcf d7acac7d5657 4300730a79f3 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.b smithi160 running (115s) 47s ago 115s 25.1M - 19.0.0-2515-g96805dcf d7acac7d5657 127928f15eec 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.c smithi188 running (2m) 47s ago 2m 16.5M - 19.0.0-2515-g96805dcf d7acac7d5657 d3ceadf87dfb 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.d smithi133 running (2m) 23s ago 2m 12.4M - 19.0.0-2515-g96805dcf d7acac7d5657 895e1076ba65 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.e smithi160 running (2m) 47s ago 2m 13.9M - 19.0.0-2515-g96805dcf d7acac7d5657 054fee29c8b0 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.f smithi188 running (2m) 47s ago 2m 15.3M - 19.0.0-2515-g96805dcf d7acac7d5657 f0ec3fa00b3f 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.g smithi133 running (2m) 23s ago 2m 16.4M - 19.0.0-2515-g96805dcf d7acac7d5657 43b3e98c0ccf 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.h smithi160 running (2m) 47s ago 2m 16.9M - 19.0.0-2515-g96805dcf d7acac7d5657 3af0fcf8920d 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.i smithi188 running (2m) 47s ago 2m 20.3M - 19.0.0-2515-g96805dcf d7acac7d5657 75e13b9ccea1 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.j smithi133 running (2m) 23s ago 2m 12.4M - 19.0.0-2515-g96805dcf d7acac7d5657 c4a2d8208e60 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.k smithi160 running (2m) 47s ago 2m 16.3M - 19.0.0-2515-g96805dcf d7acac7d5657 87c0ed76d2d0 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mds.l smithi188 running (2m) 47s ago 2m 15.5M - 19.0.0-2515-g96805dcf d7acac7d5657 acb782d122e6 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.x smithi133 *:9283,8765 running (10m) 23s ago 10m 516M - 19.0.0-2515-g96805dcf d7acac7d5657 a1a44a013c1c 2024-05-09T08:29:44.489 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.y smithi160 *:8443,8765 running (8m) 47s ago 8m 454M - 19.0.0-2515-g96805dcf d7acac7d5657 2ad8815f893c 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.z smithi188 *:8443,8765 running (8m) 47s ago 8m 454M - 19.0.0-2515-g96805dcf d7acac7d5657 98f62e89a593 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:mon.a smithi133 running (10m) 23s ago 10m 48.7M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 84f787f9bd06 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:mon.b smithi160 running (8m) 47s ago 8m 40.5M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 a1c25f7a8277 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:mon.c smithi188 running (8m) 47s ago 8m 40.4M 2048M 19.0.0-2515-g96805dcf d7acac7d5657 4b45bd867d11 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 smithi133 running (8m) 23s ago 8m 55.9M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 25bfc41f45bc 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 smithi160 running (7m) 47s ago 7m 55.2M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 2ab582a511bb 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 smithi188 running (7m) 47s ago 7m 57.9M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 c70b2c654791 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 smithi133 running (6m) 23s ago 6m 54.2M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 cdfef9052e7d 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 smithi160 running (6m) 47s ago 6m 41.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 87e4c25e21cd 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 smithi188 running (5m) 47s ago 5m 54.5M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 ef7b771206a5 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 smithi133 running (5m) 23s ago 5m 40.9M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 265ea888be61 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 smithi160 running (4m) 47s ago 4m 53.5M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 12f304b8c9f5 2024-05-09T08:29:44.490 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 smithi188 running (4m) 47s ago 4m 54.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 c5726f70ba85 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 smithi133 running (3m) 23s ago 3m 40.7M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 595d91a1422a 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 smithi160 running (3m) 47s ago 3m 44.8M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 79f4ca60ad6f 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 smithi188 running (2m) 47s ago 2m 50.3M 4272M 19.0.0-2515-g96805dcf d7acac7d5657 a67deaac4201 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.491+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 msgr2=0x7f29b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.491+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 0x7f29b00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009680 tx=0x7f29b8009260 comp rx=0 tx=0).stop 2024-05-09T08:29:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.491+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 msgr2=0x7f29c819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.491+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e950 tx=0x7f29bc00ec60 comp rx=0 tx=0).stop 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 shutdown_connections 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f29b00a2c20 0x7f29b00a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c8106160 0x7f29c81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29c81045c0 0x7f29c819a720 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 --2- 172.21.15.133:0/2158969264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c81057c0 0x7f29c819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 >> 172.21.15.133:0/2158969264 conn(0x7f29c80ffb80 msgr2=0x7f29c8108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 shutdown_connections 2024-05-09T08:29:44.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:44.492+0000 7f29d0000700 1 -- 172.21.15.133:0/2158969264 wait complete. 2024-05-09T08:29:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:44 smithi160 ceph-mon[34070]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:44 smithi160 ceph-mon[34070]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:44 smithi133 ceph-mon[29610]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:44 smithi133 ceph-mon[29610]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:44 smithi188 ceph-mon[34645]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:44 smithi188 ceph-mon[34645]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:45.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T08:29:45.766 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:47.140 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:46 smithi133 ceph-mon[29610]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:46 smithi160 ceph-mon[34070]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:46 smithi188 ceph-mon[34645]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:48.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 -- 172.21.15.133:0/2723738312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300108000 msgr2=0x7fb30010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:48.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 --2- 172.21.15.133:0/2723738312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300108000 0x7fb30010a410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b0a0 tx=0x7fb2f800b3b0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 -- 172.21.15.133:0/2723738312 shutdown_connections 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 --2- 172.21.15.133:0/2723738312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300108000 0x7fb30010a410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 --2- 172.21.15.133:0/2723738312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300102da0 0x7fb300105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 --2- 172.21.15.133:0/2723738312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 -- 172.21.15.133:0/2723738312 >> 172.21.15.133:0/2723738312 conn(0x7fb3000fc730 msgr2=0x7fb3000feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 -- 172.21.15.133:0/2723738312 shutdown_connections 2024-05-09T08:29:48.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.007+0000 7fb2ff200700 1 -- 172.21.15.133:0/2723738312 wait complete. 2024-05-09T08:29:48.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.008+0000 7fb2ff200700 1 Processor -- start 2024-05-09T08:29:48.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.008+0000 7fb2ff200700 1 -- start start 2024-05-09T08:29:48.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300102da0 0x7fb300102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:48.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 0x7fb300101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010de70 con 0x7fb3001056d0 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010dcf0 con 0x7fb300108000 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2ff200700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb30010dff0 con 0x7fb300102da0 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2fd400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2fe800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 0x7fb300101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2fd400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33230/0 (socket says 172.21.15.133:33230) 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2fd400700 1 -- 172.21.15.133:0/3846775971 learned_addr learned my addr 172.21.15.133:0/3846775971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.009+0000 7fb2fde00700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300102da0 0x7fb300102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 -- 172.21.15.133:0/3846775971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300102da0 msgr2=0x7fb300102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300102da0 0x7fb300102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 -- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 msgr2=0x7fb300100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:48.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 -- 172.21.15.133:0/3846775971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb300101b40 con 0x7fb300108000 2024-05-09T08:29:48.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fd400700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:48.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2fe800700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 0x7fb300101290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b070 tx=0x7fb2f8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:48.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8021070 con 0x7fb300108000 2024-05-09T08:29:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2f8007b80 con 0x7fb300108000 2024-05-09T08:29:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f801d450 con 0x7fb300108000 2024-05-09T08:29:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3001a24e0 con 0x7fb300108000 2024-05-09T08:29:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.010+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a2940 con 0x7fb300108000 2024-05-09T08:29:48.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.011+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb300069010 con 0x7fb300108000 2024-05-09T08:29:48.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.014+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb2f8004000 con 0x7fb300108000 2024-05-09T08:29:48.014 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.015+0000 7fb2f6a00700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 0x7fb2e40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:48.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.015+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb2f80d0340 con 0x7fb300108000 2024-05-09T08:29:48.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.015+0000 7fb2fde00700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 0x7fb2e40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:48.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.016+0000 7fb2fde00700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 0x7fb2e40a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007c60 tx=0x7fb2f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:48.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.017+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb2f809d300 con 0x7fb300108000 2024-05-09T08:29:48.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.248+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fb300062760 con 0x7fb2e40a2ca0 2024-05-09T08:29:48.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.255+0000 7fb2f6a00700 1 -- 172.21.15.133:0/3846775971 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fb300062760 con 0x7fb2e40a2ca0 2024-05-09T08:29:48.255 INFO:teuthology.orchestra.run.smithi133.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T08:29:48.255 INFO:teuthology.orchestra.run.smithi133.stdout:mds.all 12/12 51s ago 2m smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi160=b;smithi160=e;smithi160=h;smithi160=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-05-09T08:29:48.256 INFO:teuthology.orchestra.run.smithi133.stdout:mgr 3/3 51s ago 8m smithi133=x;smithi160=y;smithi188=z;count:3 2024-05-09T08:29:48.256 INFO:teuthology.orchestra.run.smithi133.stdout:mon 3/3 51s ago 9m smithi133:172.21.15.133=a;smithi160:172.21.15.160=b;smithi188:172.21.15.188=c;count:3 2024-05-09T08:29:48.256 INFO:teuthology.orchestra.run.smithi133.stdout:osd 12 51s ago - 2024-05-09T08:29:48.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 msgr2=0x7fb2e40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 0x7fb2e40a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0007c60 tx=0x7fb2f00073d0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 msgr2=0x7fb300101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 0x7fb300101290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800b070 tx=0x7fb2f8004d00 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 shutdown_connections 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb300108000 0x7fb300101290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb2e40a2ca0 0x7fb2e40a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb300102da0 0x7fb300102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 --2- 172.21.15.133:0/3846775971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3001056d0 0x7fb300100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 >> 172.21.15.133:0/3846775971 conn(0x7fb3000fc730 msgr2=0x7fb3001036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 shutdown_connections 2024-05-09T08:29:48.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:48.259+0000 7fb2ff200700 1 -- 172.21.15.133:0/3846775971 wait complete. 2024-05-09T08:29:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:48 smithi133 ceph-mon[29610]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:48 smithi133 ceph-mon[29610]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:49.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T08:29:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:48 smithi160 ceph-mon[34070]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:48 smithi160 ceph-mon[34070]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:48 smithi188 ceph-mon[34645]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:48 smithi188 ceph-mon[34645]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:49.549 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:50 smithi160 ceph-mon[34070]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:50 smithi133 ceph-mon[29610]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:50 smithi188 ceph-mon[34645]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:51.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 -- 172.21.15.133:0/768033338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1056c0 msgr2=0x7f133c105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 --2- 172.21.15.133:0/768033338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1056c0 0x7f133c105b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13280099c0 tx=0x7f1328009cd0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 -- 172.21.15.133:0/768033338 shutdown_connections 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 --2- 172.21.15.133:0/768033338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 --2- 172.21.15.133:0/768033338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1044c0 0x7f133c1048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 --2- 172.21.15.133:0/768033338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1056c0 0x7f133c105b20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 -- 172.21.15.133:0/768033338 >> 172.21.15.133:0/768033338 conn(0x7f133c0ffa40 msgr2=0x7f133c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.723+0000 7f1342e00700 1 -- 172.21.15.133:0/768033338 shutdown_connections 2024-05-09T08:29:51.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.724+0000 7f1342e00700 1 -- 172.21.15.133:0/768033338 wait complete. 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 Processor -- start 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 -- start start 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 0x7f133c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:51.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1100b0 con 0x7f133c1044c0 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f133c10ff30 con 0x7f133c1056c0 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.725+0000 7f1342e00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f133c110230 con 0x7f133c106060 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 0x7f133c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33250/0 (socket says 172.21.15.133:33250) 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 -- 172.21.15.133:0/510118308 learned_addr learned my addr 172.21.15.133:0/510118308 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:51.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1342400700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 -- 172.21.15.133:0/510118308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 msgr2=0x7f133c1a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c1a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 -- 172.21.15.133:0/510118308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 msgr2=0x7f133c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 0x7f133c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 -- 172.21.15.133:0/510118308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a2290 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1342400700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341000700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 0x7f133c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.726+0000 7f1341a00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9a0 tx=0x7f133000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.727+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330004220 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.727+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13300043c0 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.727+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330003df0 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.727+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a2580 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.727+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a2a70 con 0x7f133c1044c0 2024-05-09T08:29:51.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.728+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c069010 con 0x7f133c1044c0 2024-05-09T08:29:51.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.730+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f133000c810 con 0x7f133c1044c0 2024-05-09T08:29:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.732+0000 7f133a000700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.732+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f13300cf1d0 con 0x7f133c1044c0 2024-05-09T08:29:51.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.732+0000 7f1341000700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:51.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.733+0000 7f1341000700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 0x7f13240a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1328000c00 tx=0x7f1328005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:51.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.736+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f133009c100 con 0x7f133c1044c0 2024-05-09T08:29:51.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.966+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f133c062760 con 0x7f13240a2ca0 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.969+0000 7f133a000700 1 -- 172.21.15.133:0/510118308 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f133c062760 con 0x7f13240a2ca0 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stdout:HOST ADDR LABELS STATUS 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 172.21.15.133 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stdout:smithi160 172.21.15.160 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stdout:smithi188 172.21.15.188 2024-05-09T08:29:51.969 INFO:teuthology.orchestra.run.smithi133.stdout:3 hosts in cluster 2024-05-09T08:29:51.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 msgr2=0x7f13240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:51.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 0x7f13240a5160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1328000c00 tx=0x7f1328005fb0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 msgr2=0x7f133c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:51.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9a0 tx=0x7f133000ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 shutdown_connections 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f133c106060 0x7f133c1a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f133c1044c0 0x7f133c19a630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 --2- 172.21.15.133:0/510118308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f133c1056c0 0x7f133c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 >> 172.21.15.133:0/510118308 conn(0x7f133c0ffa40 msgr2=0x7f133c108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 shutdown_connections 2024-05-09T08:29:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:51.972+0000 7f1342e00700 1 -- 172.21.15.133:0/510118308 wait complete. 2024-05-09T08:29:52.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T08:29:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:52 smithi160 ceph-mon[34070]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:53.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:52 smithi160 ceph-mon[34070]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:52 smithi133 ceph-mon[29610]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:52 smithi133 ceph-mon[29610]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:52 smithi188 ceph-mon[34645]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:52 smithi188 ceph-mon[34645]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:53.234 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:54.916 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:54 smithi133 ceph-mon[29610]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:54 smithi160 ceph-mon[34070]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:54 smithi188 ceph-mon[34645]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 -- 172.21.15.133:0/3828499662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8102390 msgr2=0x7f4ec81027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/3828499662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8102390 0x7f4ec81027b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00099c0 tx=0x7f4eb0009cd0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 -- 172.21.15.133:0/3828499662 shutdown_connections 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/3828499662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103f30 0x7f4ec810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/3828499662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8102390 0x7f4ec81027b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/3828499662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8103590 0x7f4ec81039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 -- 172.21.15.133:0/3828499662 >> 172.21.15.133:0/3828499662 conn(0x7f4ec80fd8f0 msgr2=0x7f4ec80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 -- 172.21.15.133:0/3828499662 shutdown_connections 2024-05-09T08:29:55.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.452+0000 7f4ecfa00700 1 -- 172.21.15.133:0/3828499662 wait complete. 2024-05-09T08:29:55.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.453+0000 7f4ecfa00700 1 Processor -- start 2024-05-09T08:29:55.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 -- start start 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8102390 0x7f4ec80760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8103f30 0x7f4ec8074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec810dfa0 con 0x7f4ec8103f30 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec810de20 con 0x7f4ec8102390 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecfa00700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec810e120 con 0x7f4ec8103590 2024-05-09T08:29:55.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ebfe00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:55.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ebfe00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:52792/0 (socket says 172.21.15.133:52792) 2024-05-09T08:29:55.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ebfe00700 1 -- 172.21.15.133:0/2781004166 learned_addr learned my addr 172.21.15.133:0/2781004166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:55.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecda00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8103f30 0x7f4ec8074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:55.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.454+0000 7f4ecd000700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8102390 0x7f4ec80760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:55.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 -- 172.21.15.133:0/2781004166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8102390 msgr2=0x7f4ec80760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8102390 0x7f4ec80760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 -- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8103f30 msgr2=0x7f4ec8074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8103f30 0x7f4ec8074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 -- 172.21.15.133:0/2781004166 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec8075500 con 0x7f4ec8103590 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ebfe00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9a0 tx=0x7f4eb800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.455+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800c9c0 con 0x7f4ec8103590 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.456+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb800cb60 con 0x7f4ec8103590 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.456+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec81a2770 con 0x7f4ec8103590 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.456+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8018660 con 0x7f4ec8103590 2024-05-09T08:29:55.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.456+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a2cb0 con 0x7f4ec8103590 2024-05-09T08:29:55.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.457+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec8069010 con 0x7f4ec8103590 2024-05-09T08:29:55.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.459+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4eb800cd10 con 0x7f4ec8103590 2024-05-09T08:29:55.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.460+0000 7f4ec6000700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 0x7f4ea80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:55.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.460+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4eb80cf4e0 con 0x7f4ec8103590 2024-05-09T08:29:55.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.460+0000 7f4ecd000700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 0x7f4ea80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:55.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.461+0000 7f4ecd000700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 0x7f4ea80a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b580 tx=0x7f4eb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:55.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.463+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4eb809c410 con 0x7f4ec8103590 2024-05-09T08:29:55.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.694+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4ec8062760 con 0x7f4ea80a2ca0 2024-05-09T08:29:55.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.697+0000 7f4ec6000700 1 -- 172.21.15.133:0/2781004166 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4ec8062760 con 0x7f4ea80a2ca0 2024-05-09T08:29:55.697 INFO:teuthology.orchestra.run.smithi133.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T08:29:55.697 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T08:29:55.697 INFO:teuthology.orchestra.run.smithi133.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-09T08:29:55.697 INFO:teuthology.orchestra.run.smithi133.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 msgr2=0x7f4ea80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 0x7f4ea80a5160 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b580 tx=0x7f4eb0005fb0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 msgr2=0x7f4ec8074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9a0 tx=0x7f4eb800ecb0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 shutdown_connections 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec8103f30 0x7f4ec8074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ec8102390 0x7f4ec80760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4ea80a2ca0 0x7f4ea80a5160 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 --2- 172.21.15.133:0/2781004166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec8103590 0x7f4ec8074710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:55.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 >> 172.21.15.133:0/2781004166 conn(0x7f4ec80fd8f0 msgr2=0x7f4ec8106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:55.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 shutdown_connections 2024-05-09T08:29:55.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:55.700+0000 7f4ecfa00700 1 -- 172.21.15.133:0/2781004166 wait complete. 2024-05-09T08:29:56.083 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:55 smithi133 ceph-mon[29610]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:56.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:55 smithi160 ceph-mon[34070]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:56.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:55 smithi188 ceph-mon[34645]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:29:56.533 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T08:29:56.542 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T08:29:56.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T08:29:57.002 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:29:57.151 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:56 smithi133 ceph-mon[29610]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:56 smithi160 ceph-mon[34070]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:57.183 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:56 smithi188 ceph-mon[34645]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:29:59.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 -- 172.21.15.133:0/159028273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc81057d0 msgr2=0x7f2cc8105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:59.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 --2- 172.21.15.133:0/159028273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc81057d0 0x7f2cc8105c30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40099c0 tx=0x7f2cb4009cd0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.086 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:58 smithi133 ceph-mon[29610]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 -- 172.21.15.133:0/159028273 shutdown_connections 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 --2- 172.21.15.133:0/159028273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc8106170 0x7f2cc810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 --2- 172.21.15.133:0/159028273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 --2- 172.21.15.133:0/159028273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc81057d0 0x7f2cc8105c30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 -- 172.21.15.133:0/159028273 >> 172.21.15.133:0/159028273 conn(0x7f2cc80ffb70 msgr2=0x7f2cc8101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:59.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.085+0000 7f2cce000700 1 -- 172.21.15.133:0/159028273 shutdown_connections 2024-05-09T08:29:59.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.086+0000 7f2cce000700 1 -- 172.21.15.133:0/159028273 wait complete. 2024-05-09T08:29:59.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 Processor -- start 2024-05-09T08:29:59.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 -- start start 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 0x7f2cc819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc8106170 0x7f2cc81a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8110210 con 0x7f2cc8106170 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8110090 con 0x7f2cc81045d0 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.087+0000 7f2cce000700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8110390 con 0x7f2cc81057d0 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:59.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc6a00700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 0x7f2cc819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:56094/0 (socket says 172.21.15.133:56094) 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 -- 172.21.15.133:0/1508992487 learned_addr learned my addr 172.21.15.133:0/1508992487 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc8106170 0x7f2cc81a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 -- 172.21.15.133:0/1508992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 msgr2=0x7f2cc819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 0x7f2cc819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 -- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc8106170 msgr2=0x7f2cc81a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc8106170 0x7f2cc81a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 -- 172.21.15.133:0/1508992487 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc81a23d0 con 0x7f2cc81045d0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc6a00700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 0x7f2cc819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.088+0000 7f2cc7400700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d980 tx=0x7f2cb800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.089+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8016070 con 0x7f2cc81045d0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.089+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb800fd70 con 0x7f2cc81045d0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.089+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc81a26c0 con 0x7f2cc81045d0 2024-05-09T08:29:59.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.089+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8019360 con 0x7f2cc81045d0 2024-05-09T08:29:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.089+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81a2c00 con 0x7f2cc81045d0 2024-05-09T08:29:59.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.090+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc8069010 con 0x7f2cc81045d0 2024-05-09T08:29:59.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.092+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2cb8019070 con 0x7f2cc81045d0 2024-05-09T08:29:59.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.093+0000 7f2cbfe00700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 0x7f2cac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:29:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.093+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2cb80ce700 con 0x7f2cc81045d0 2024-05-09T08:29:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.093+0000 7f2cc6a00700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 0x7f2cac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:29:59.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.094+0000 7f2cc6a00700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 0x7f2cac0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4000c00 tx=0x7f2cb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:29:59.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.097+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2cb809b630 con 0x7f2cc81045d0 2024-05-09T08:29:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:58 smithi160 ceph-mon[34070]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:58 smithi188 ceph-mon[34645]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:29:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.365+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2cc8062f30 con 0x7f2cc81045d0 2024-05-09T08:29:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.366+0000 7f2cbfe00700 1 -- 172.21.15.133:0/1508992487 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) ==== 76+0+3062 (secure 0 0 0) 0x7f2cb809af30 con 0x7f2cc81045d0 2024-05-09T08:29:59.366 INFO:teuthology.orchestra.run.smithi133.stdout:e22 2024-05-09T08:29:59.366 INFO:teuthology.orchestra.run.smithi133.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:legacy client fscid: 1 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:Filesystem 'cephfs' (1) 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:fs_name cephfs 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 22 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-05-09T08:27:55.241118+0000 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2024-05-09T08:28:01.270664+0000 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:tableserver 0 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:root 0 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:session_timeout 300 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:session_autoclose 300 2024-05-09T08:29:59.367 INFO:teuthology.orchestra.run.smithi133.stdout:max_file_size 1099511627776 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:max_xattr_size 65536 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:required_client_features {} 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure 0 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure_osd_epoch 0 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:max_mds 3 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:in 0,1,2 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:up {0=24566,1=24556,2=24536} 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:failed 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:damaged 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:stopped 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:data_pools [3] 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:metadata_pool 2 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:inline_data disabled 2024-05-09T08:29:59.368 INFO:teuthology.orchestra.run.smithi133.stdout:balancer 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:bal_rank_mask -1 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:standby_count_wanted 0 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:qdb_cluster leader: 24566 members: 24536,24556,24566 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b{0:24566} state up:active seq 3 addr [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.i{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.188:6838/2528648460,v1:172.21.15.188:6839/2528648460] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a{1:24556} state up:active seq 5 addr [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.h{1:24509} state up:standby-replay seq 1 addr [v2:172.21.15.160:6836/1342622403,v1:172.21.15.160:6837/1342622403] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.g{2:24536} state up:active seq 6 addr [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.188:6836/3112088995,v1:172.21.15.188:6837/3112088995] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:Standby daemons: 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.133:6834/1799489914,v1:172.21.15.133:6835/1799489914] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.369 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d{-1:14394} state up:standby seq 1 addr [v2:172.21.15.133:6836/3539634696,v1:172.21.15.133:6837/3539634696] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.188:6832/1493326462,v1:172.21.15.188:6833/1493326462] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.160:6832/4109875508,v1:172.21.15.160:6833/4109875508] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e{-1:24482} state up:standby seq 1 addr [v2:172.21.15.160:6834/4102808369,v1:172.21.15.160:6835/4102808369] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f{-1:24493} state up:standby seq 1 addr [v2:172.21.15.188:6834/232300149,v1:172.21.15.188:6835/232300149] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 msgr2=0x7f2cac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 0x7f2cac0a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4000c00 tx=0x7f2cb4005fb0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 msgr2=0x7f2cc819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:29:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d980 tx=0x7f2cb800dc90 comp rx=0 tx=0).stop 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 shutdown_connections 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cc8106170 0x7f2cc81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2cc81045d0 0x7f2cc819a770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2cac0a2c20 0x7f2cac0a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 --2- 172.21.15.133:0/1508992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cc81057d0 0x7f2cc819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 >> 172.21.15.133:0/1508992487 conn(0x7f2cc80ffb70 msgr2=0x7f2cc8108a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 shutdown_connections 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:29:59.370+0000 7f2cce000700 1 -- 172.21.15.133:0/1508992487 wait complete. 2024-05-09T08:29:59.371 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:30:00.102 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:29:59 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1508992487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T08:30:00.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:29:59 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1508992487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T08:30:00.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:29:59 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1508992487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T08:30:00.184 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T08:30:00.631 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:30:01.114 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:00 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:30:01.115 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:00 smithi133 ceph-mon[29610]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:00 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:30:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:00 smithi160 ceph-mon[34070]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:00 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:30:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:00 smithi188 ceph-mon[34645]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 -- 172.21.15.133:0/1698416158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c106160 msgr2=0x7f833c10ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 --2- 172.21.15.133:0/1698416158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c106160 0x7f833c10ca10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f833400b0a0 tx=0x7f833400b3b0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 -- 172.21.15.133:0/1698416158 shutdown_connections 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 --2- 172.21.15.133:0/1698416158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c106160 0x7f833c10ca10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 --2- 172.21.15.133:0/1698416158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1045c0 0x7f833c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 --2- 172.21.15.133:0/1698416158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c1057c0 0x7f833c105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.735+0000 7f8341800700 1 -- 172.21.15.133:0/1698416158 >> 172.21.15.133:0/1698416158 conn(0x7f833c0ffb80 msgr2=0x7f833c101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.736+0000 7f8341800700 1 -- 172.21.15.133:0/1698416158 shutdown_connections 2024-05-09T08:30:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.736+0000 7f8341800700 1 -- 172.21.15.133:0/1698416158 wait complete. 2024-05-09T08:30:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 Processor -- start 2024-05-09T08:30:02.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 -- start start 2024-05-09T08:30:02.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c1045c0 0x7f833c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c106160 0x7f833c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1101b0 con 0x7f833c1057c0 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f833c110030 con 0x7f833c106160 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f8341800700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f833c110330 con 0x7f833c1045c0 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f833a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f833a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47488/0 (socket says 172.21.15.133:47488) 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.737+0000 7f833a000700 1 -- 172.21.15.133:0/3813135997 learned_addr learned my addr 172.21.15.133:0/3813135997 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833aa00700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c1045c0 0x7f833c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833b400700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c106160 0x7f833c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 -- 172.21.15.133:0/3813135997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c1045c0 msgr2=0x7f833c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c1045c0 0x7f833c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 -- 172.21.15.133:0/3813135997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c106160 msgr2=0x7f833c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c106160 0x7f833c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 -- 172.21.15.133:0/3813135997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c1a2390 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f833a000700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f832400e9f0 tx=0x7f832400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.738+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400cbd0 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.739+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c1a2680 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.739+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a2bc0 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.739+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832400ee90 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.739+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83240056c0 con 0x7f833c1057c0 2024-05-09T08:30:02.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.740+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8318005360 con 0x7f833c1057c0 2024-05-09T08:30:02.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.743+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8324004750 con 0x7f833c1057c0 2024-05-09T08:30:02.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.745+0000 7f8333400700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.745+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f83240cf390 con 0x7f833c1057c0 2024-05-09T08:30:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.745+0000 7f833aa00700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:02.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.746+0000 7f833aa00700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 0x7f83200a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f833c105620 tx=0x7f832c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:02.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.747+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8324053740 con 0x7f833c1057c0 2024-05-09T08:30:02.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:02 smithi133 ceph-mon[29610]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:02.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.977+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f8318005920 con 0x7f833c1057c0 2024-05-09T08:30:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.979+0000 7f8333400700 1 -- 172.21.15.133:0/3813135997 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5284 (secure 0 0 0) 0x7f832409c2c0 con 0x7f833c1057c0 2024-05-09T08:30:02.979 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 88 2024-05-09T08:30:02.979 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-05-09T08:18:49.174777+0000 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2024-05-09T08:28:57.810205+0000 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:crush_version 26 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:full_ratio 0.95 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:backfillfull_ratio 0.9 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:nearfull_ratio 0.85 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:require_min_compat_client luminous 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:min_compat_client luminous 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:require_osd_release squid 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:stretch_mode_enabled false 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T08:30:02.980 INFO:teuthology.orchestra.run.smithi133.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:max_osd 12 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6802/2872423449,v1:172.21.15.133:6803/2872423449] [v2:172.21.15.133:6804/2872423449,v1:172.21.15.133:6805/2872423449] exists,up d560e467-02ea-4321-815b-434186255b78 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/2987709381,v1:172.21.15.160:6801/2987709381] [v2:172.21.15.160:6802/2987709381,v1:172.21.15.160:6803/2987709381] exists,up 082e4943-4828-4bdd-a221-337d3c05a7ee 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/2504154225,v1:172.21.15.188:6801/2504154225] [v2:172.21.15.188:6802/2504154225,v1:172.21.15.188:6803/2504154225] exists,up 228eae91-e83f-4241-ae1f-a4f38890d2db 2024-05-09T08:30:02.981 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6810/2675627022,v1:172.21.15.133:6811/2675627022] [v2:172.21.15.133:6812/2675627022,v1:172.21.15.133:6813/2675627022] exists,up c20bfe2a-6caf-485b-b3d2-4d661db1ffef 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/1804660056,v1:172.21.15.160:6809/1804660056] [v2:172.21.15.160:6810/1804660056,v1:172.21.15.160:6811/1804660056] exists,up ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/485185361,v1:172.21.15.188:6809/485185361] [v2:172.21.15.188:6810/485185361,v1:172.21.15.188:6811/485185361] exists,up 41748ba8-d09c-4d93-a815-11ef99a89358 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6818/1108533027,v1:172.21.15.133:6819/1108533027] [v2:172.21.15.133:6820/1108533027,v1:172.21.15.133:6821/1108533027] exists,up cc1a882a-80f7-4af1-954d-4005e2b92111 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/3423087941,v1:172.21.15.160:6817/3423087941] [v2:172.21.15.160:6818/3423087941,v1:172.21.15.160:6819/3423087941] exists,up 3f4a0b43-4191-4999-93e4-929b414f0822 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/4036931105,v1:172.21.15.188:6817/4036931105] [v2:172.21.15.188:6818/4036931105,v1:172.21.15.188:6819/4036931105] exists,up 0fcbb1ac-8999-488b-965a-a1d6236162f3 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6826/3729207054,v1:172.21.15.133:6827/3729207054] [v2:172.21.15.133:6828/3729207054,v1:172.21.15.133:6829/3729207054] exists,up af5f780d-dfa5-4680-a6cf-c821033f7540 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/993185215,v1:172.21.15.160:6825/993185215] [v2:172.21.15.160:6826/993185215,v1:172.21.15.160:6827/993185215] exists,up 83143e78-aa23-47d6-836a-3fd438f33c0d 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/2170090955,v1:172.21.15.188:6825/2170090955] [v2:172.21.15.188:6826/2170090955,v1:172.21.15.188:6827/2170090955] exists,up 7d934675-6b21-4c54-ac51-12593da6e97e 2024-05-09T08:30:02.982 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.a [2,11] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.b [8,11] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.0 [4,8] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.c [5,6] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.2c [4,9] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.36 [4,9] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.38 [3,9] 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/3643547543 expires 2024-05-10T08:19:56.515544+0000 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/3643547543 expires 2024-05-10T08:19:56.515544+0000 2024-05-09T08:30:02.983 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/899667592 expires 2024-05-10T08:19:56.515544+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/2316286583 expires 2024-05-10T08:19:56.515544+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/1186224387 expires 2024-05-10T08:19:22.399959+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/74425247 expires 2024-05-10T08:19:22.399959+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/1186224387 expires 2024-05-10T08:19:22.399959+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/775453271 expires 2024-05-10T08:19:22.399959+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/2601959981 expires 2024-05-10T08:19:22.399959+0000 2024-05-09T08:30:02.984 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/3973005658 expires 2024-05-10T08:19:56.515544+0000 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.983+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 msgr2=0x7f83200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.983+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 0x7f83200a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f833c105620 tx=0x7f832c009260 comp rx=0 tx=0).stop 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.983+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 msgr2=0x7f833c19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.983+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f832400e9f0 tx=0x7f832400ed00 comp rx=0 tx=0).stop 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 shutdown_connections 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f833c106160 0x7f833c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f83200a2ca0 0x7f83200a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f833c1045c0 0x7f833c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 --2- 172.21.15.133:0/3813135997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c1057c0 0x7f833c19ac70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:02.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 >> 172.21.15.133:0/3813135997 conn(0x7f833c0ffb80 msgr2=0x7f833c108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:02.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 shutdown_connections 2024-05-09T08:30:02.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:02.984+0000 7f8341800700 1 -- 172.21.15.133:0/3813135997 wait complete. 2024-05-09T08:30:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:02 smithi160 ceph-mon[34070]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:02 smithi188 ceph-mon[34645]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:03.772 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T08:30:03.782 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.988+0000 7f0494e00640 1 -- 172.21.15.133:0/4126060232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490100880 msgr2=0x7f0490100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.988+0000 7f0494e00640 1 --2- 172.21.15.133:0/4126060232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490100880 0x7f0490100ce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0478009860 tx=0x7f047802a340 comp rx=0 tx=0).stop 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 -- 172.21.15.133:0/4126060232 shutdown_connections 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 --2- 172.21.15.133:0/4126060232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0490101220 0x7f04901016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 --2- 172.21.15.133:0/4126060232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490100880 0x7f0490100ce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 --2- 172.21.15.133:0/4126060232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04900fff20 0x7f0490100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 -- 172.21.15.133:0/4126060232 >> 172.21.15.133:0/4126060232 conn(0x7f04900fbb40 msgr2=0x7f04900fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 -- 172.21.15.133:0/4126060232 shutdown_connections 2024-05-09T08:30:03.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.989+0000 7f0494e00640 1 -- 172.21.15.133:0/4126060232 wait complete. 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.990+0000 7f0494e00640 1 Processor -- start 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.990+0000 7f0494e00640 1 -- start start 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.990+0000 7f0494e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.990+0000 7f0494e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 0x7f049019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0494e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490101220 0x7f04901a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0494e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f049010f520 con 0x7f0490100880 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0494e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f049010f3a0 con 0x7f0490101220 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0494e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f049010f6a0 con 0x7f04900fff20 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0486a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 0x7f049019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:03.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:58700/0 (socket says 172.21.15.133:58700) 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 -- 172.21.15.133:0/4239244950 learned_addr learned my addr 172.21.15.133:0/4239244950 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048fe00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490101220 0x7f04901a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 -- 172.21.15.133:0/4239244950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490101220 msgr2=0x7f04901a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490101220 0x7f04901a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 -- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 msgr2=0x7f049019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 0x7f049019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f048f400640 1 -- 172.21.15.133:0/4239244950 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04901a17c0 con 0x7f04900fff20 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.991+0000 7f0486a00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 0x7f049019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f048f400640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f048000e9c0 tx=0x7f048000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048000ccc0 con 0x7f04900fff20 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0480005500 con 0x7f04900fff20 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04901a1ab0 con 0x7f04900fff20 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480010430 con 0x7f04900fff20 2024-05-09T08:30:03.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.992+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901a2020 con 0x7f04900fff20 2024-05-09T08:30:03.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.993+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049010ac70 con 0x7f04900fff20 2024-05-09T08:30:03.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.995+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f04800046c0 con 0x7f04900fff20 2024-05-09T08:30:03.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.996+0000 7f048cc00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 0x7f04740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:03.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.997+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f04800cf2d0 con 0x7f04900fff20 2024-05-09T08:30:03.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.997+0000 7f0486a00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 0x7f04740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:03.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:03.997+0000 7f0486a00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 0x7f04740a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f047802a930 tx=0x7f0478030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:04.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.001+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f048009c200 con 0x7f04900fff20 2024-05-09T08:30:04.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:03 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3813135997' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T08:30:04.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:03 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3813135997' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T08:30:04.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:03 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3813135997' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T08:30:04.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.202+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04900624d0 con 0x7f04900fff20 2024-05-09T08:30:04.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.204+0000 7f048cc00640 1 -- 172.21.15.133:0/4239244950 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7f048009bb00 con 0x7f04900fff20 2024-05-09T08:30:04.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:30:04.205 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 msgr2=0x7f04740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 0x7f04740a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f047802a930 tx=0x7f0478030570 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 msgr2=0x7f0490199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f048000e9c0 tx=0x7f048000ee80 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 shutdown_connections 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04740a2c10 0x7f04740a50d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0490101220 0x7f04901a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0490100880 0x7f049019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 --2- 172.21.15.133:0/4239244950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04900fff20 0x7f0490199ba0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 >> 172.21.15.133:0/4239244950 conn(0x7f04900fbb40 msgr2=0x7f04900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:04.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 shutdown_connections 2024-05-09T08:30:04.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.208+0000 7f0494e00640 1 -- 172.21.15.133:0/4239244950 wait complete. 2024-05-09T08:30:04.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:30:04.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 -- 172.21.15.133:0/4074786401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760103a90 msgr2=0x7f9760103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 --2- 172.21.15.133:0/4074786401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760103a90 0x7f9760103eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9754009860 tx=0x7f975402a340 comp rx=0 tx=0).stop 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 -- 172.21.15.133:0/4074786401 shutdown_connections 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 --2- 172.21.15.133:0/4074786401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760105640 0x7f976010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 --2- 172.21.15.133:0/4074786401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 0x7f9760105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 --2- 172.21.15.133:0/4074786401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760103a90 0x7f9760103eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 -- 172.21.15.133:0/4074786401 >> 172.21.15.133:0/4074786401 conn(0x7f97600ff020 msgr2=0x7f9760101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 -- 172.21.15.133:0/4074786401 shutdown_connections 2024-05-09T08:30:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.437+0000 7f9767600640 1 -- 172.21.15.133:0/4074786401 wait complete. 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.438+0000 7f9767600640 1 Processor -- start 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.438+0000 7f9767600640 1 -- start start 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 0x7f9760199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 0x7f976019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f976010f6e0 con 0x7f9760103a90 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f976010f560 con 0x7f9760105640 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9767600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f976010f860 con 0x7f9760104c80 2024-05-09T08:30:04.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9764c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 0x7f9760199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37094/0 (socket says 172.21.15.133:37094) 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 -- 172.21.15.133:0/149175020 learned_addr learned my addr 172.21.15.133:0/149175020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f975fe00640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 0x7f976019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 -- 172.21.15.133:0/149175020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 msgr2=0x7f976019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 0x7f976019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 -- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 msgr2=0x7f9760199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 0x7f9760199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9765600640 1 -- 172.21.15.133:0/149175020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a1a30 con 0x7f9760105640 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.439+0000 7f9764c00640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 0x7f9760199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f9765600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f975800eb10 tx=0x7f975800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800ce20 con 0x7f9760105640 2024-05-09T08:30:04.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758005020 con 0x7f9760105640 2024-05-09T08:30:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758010690 con 0x7f9760105640 2024-05-09T08:30:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a1d20 con 0x7f9760105640 2024-05-09T08:30:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.440+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a21c0 con 0x7f9760105640 2024-05-09T08:30:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.441+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976010ae20 con 0x7f9760105640 2024-05-09T08:30:04.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.443+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9758005540 con 0x7f9760105640 2024-05-09T08:30:04.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.445+0000 7f975d600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 0x7f97480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.445+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f97580cf5b0 con 0x7f9760105640 2024-05-09T08:30:04.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.445+0000 7f9764c00640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 0x7f97480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.445+0000 7f9764c00640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 0x7f97480a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9754009830 tx=0x7f9754030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:04.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.447+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f975809c4e0 con 0x7f9760105640 2024-05-09T08:30:04.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.649+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97600624d0 con 0x7f9760105640 2024-05-09T08:30:04.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.651+0000 7f975d600640 1 -- 172.21.15.133:0/149175020 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f975809bde0 con 0x7f9760105640 2024-05-09T08:30:04.650 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 msgr2=0x7f97480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 0x7f97480a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9754009830 tx=0x7f9754030570 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 msgr2=0x7f97601a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f975800eb10 tx=0x7f975800ef10 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 shutdown_connections 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97480a2b90 0x7f97480a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9760105640 0x7f97601a13c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9760104c80 0x7f976019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 --2- 172.21.15.133:0/149175020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9760103a90 0x7f9760199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 >> 172.21.15.133:0/149175020 conn(0x7f97600ff020 msgr2=0x7f9760107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:04.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 shutdown_connections 2024-05-09T08:30:04.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.654+0000 7f9767600640 1 -- 172.21.15.133:0/149175020 wait complete. 2024-05-09T08:30:04.675 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 -- 172.21.15.133:0/164188577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500761f0 msgr2=0x7f2550076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 --2- 172.21.15.133:0/164188577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500761f0 0x7f2550076610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2540009860 tx=0x7f254002a340 comp rx=0 tx=0).stop 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 -- 172.21.15.133:0/164188577 shutdown_connections 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 --2- 172.21.15.133:0/164188577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077dd0 0x7f2550078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 --2- 172.21.15.133:0/164188577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2550077430 0x7f2550077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 --2- 172.21.15.133:0/164188577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500761f0 0x7f2550076610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 -- 172.21.15.133:0/164188577 >> 172.21.15.133:0/164188577 conn(0x7f25500ff020 msgr2=0x7f2550101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 -- 172.21.15.133:0/164188577 shutdown_connections 2024-05-09T08:30:04.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.882+0000 7f2556a00640 1 -- 172.21.15.133:0/164188577 wait complete. 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.883+0000 7f2556a00640 1 Processor -- start 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.883+0000 7f2556a00640 1 -- start start 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077430 0x7f255019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2550077dd0 0x7f25501a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2550113ae0 con 0x7f25500761f0 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2550113960 con 0x7f2550077430 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2556a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2550113c60 con 0x7f2550077dd0 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48184/0 (socket says 172.21.15.133:48184) 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 -- 172.21.15.133:0/1410791836 learned_addr learned my addr 172.21.15.133:0/1410791836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:04.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2547400640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077430 0x7f255019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f2554a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2550077dd0 0x7f25501a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 -- 172.21.15.133:0/1410791836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2550077dd0 msgr2=0x7f25501a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2550077dd0 0x7f25501a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 -- 172.21.15.133:0/1410791836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077430 msgr2=0x7f255019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077430 0x7f255019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.884+0000 7f254fe00640 1 -- 172.21.15.133:0/1410791836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25501a5e10 con 0x7f25500761f0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f254fe00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2540035040 tx=0x7f2540033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540037040 con 0x7f25500761f0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540030850 con 0x7f25500761f0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501a60a0 con 0x7f25500761f0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540004920 con 0x7f25500761f0 2024-05-09T08:30:04.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.885+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501a6590 con 0x7f25500761f0 2024-05-09T08:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.886+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2550110400 con 0x7f25500761f0 2024-05-09T08:30:04.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.888+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2540004050 con 0x7f25500761f0 2024-05-09T08:30:04.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.890+0000 7f254d600640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 0x7f25340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:04.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.890+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f25400eefe0 con 0x7f25500761f0 2024-05-09T08:30:04.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.890+0000 7f2547400640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 0x7f25340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:04.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.890+0000 7f2547400640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 0x7f25340a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f255019f700 tx=0x7f2538009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:04.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:04.894+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f25400bbf10 con 0x7f25500761f0 2024-05-09T08:30:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:04 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4239244950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:04 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/149175020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:04.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:04 smithi133 ceph-mon[29610]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.094+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2550062430 con 0x7f25500761f0 2024-05-09T08:30:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.095+0000 7f254d600640 1 -- 172.21.15.133:0/1410791836 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f25400bb810 con 0x7f25500761f0 2024-05-09T08:30:05.095 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 msgr2=0x7f25340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 0x7f25340a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f255019f700 tx=0x7f2538009260 comp rx=0 tx=0).stop 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 msgr2=0x7f255019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2540035040 tx=0x7f2540033d90 comp rx=0 tx=0).stop 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 shutdown_connections 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f25340a2b90 0x7f25340a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2550077dd0 0x7f25501a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2550077430 0x7f255019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 --2- 172.21.15.133:0/1410791836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25500761f0 0x7f255019e1f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 >> 172.21.15.133:0/1410791836 conn(0x7f25500ff020 msgr2=0x7f2550100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:05.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 shutdown_connections 2024-05-09T08:30:05.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.098+0000 7f2556a00640 1 -- 172.21.15.133:0/1410791836 wait complete. 2024-05-09T08:30:05.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:30:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:04 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4239244950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:04 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/149175020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:04 smithi160 ceph-mon[34070]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:04 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4239244950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:04 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/149175020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:04 smithi188 ceph-mon[34645]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 -- 172.21.15.133:0/1126652935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858101880 msgr2=0x7f5858101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 --2- 172.21.15.133:0/1126652935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858101880 0x7f5858101ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5848009900 tx=0x7f584802a410 comp rx=0 tx=0).stop 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 -- 172.21.15.133:0/1126652935 shutdown_connections 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 --2- 172.21.15.133:0/1126652935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858103430 0x7f5858109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 --2- 172.21.15.133:0/1126652935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858102a70 0x7f5858102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 --2- 172.21.15.133:0/1126652935 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858101880 0x7f5858101ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 -- 172.21.15.133:0/1126652935 >> 172.21.15.133:0/1126652935 conn(0x7f58580fce10 msgr2=0x7f58580ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 -- 172.21.15.133:0/1126652935 shutdown_connections 2024-05-09T08:30:05.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.320+0000 7f585ee00640 1 -- 172.21.15.133:0/1126652935 wait complete. 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.321+0000 7f585ee00640 1 Processor -- start 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.321+0000 7f585ee00640 1 -- start start 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 0x7f5858073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858103430 0x7f5858074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d550 con 0x7f5858103430 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d3d0 con 0x7f5858102a70 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f585810d6d0 con 0x7f5858101880 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:58758/0 (socket says 172.21.15.133:58758) 2024-05-09T08:30:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 -- 172.21.15.133:0/3233377735 learned_addr learned my addr 172.21.15.133:0/3233377735 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f585ce00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858103430 0x7f5858074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f584f400640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 0x7f5858073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 -- 172.21.15.133:0/3233377735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 msgr2=0x7f5858073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 0x7f5858073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 -- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858103430 msgr2=0x7f5858074160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858103430 0x7f5858074160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f5857e00640 1 -- 172.21.15.133:0/3233377735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580749f0 con 0x7f5858101880 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.322+0000 7f584f400640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 0x7f5858073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f5857e00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5848031440 tx=0x7f5848032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584803f030 con 0x7f5858101880 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5848003ce0 con 0x7f5858101880 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58581a1c50 con 0x7f5858101880 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848035070 con 0x7f5858101880 2024-05-09T08:30:05.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.323+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581a21a0 con 0x7f5858101880 2024-05-09T08:30:05.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.324+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858108c20 con 0x7f5858101880 2024-05-09T08:30:05.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.326+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5848042030 con 0x7f5858101880 2024-05-09T08:30:05.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.327+0000 7f5855600640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 0x7f583c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.328+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f58480ef8b0 con 0x7f5858101880 2024-05-09T08:30:05.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.328+0000 7f584f400640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 0x7f583c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.328+0000 7f584f400640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 0x7f583c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5858075350 tx=0x7f5840005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:05.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.332+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58480bc7e0 con 0x7f5858101880 2024-05-09T08:30:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.533+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5858062430 con 0x7f5858101880 2024-05-09T08:30:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.534+0000 7f5855600640 1 -- 172.21.15.133:0/3233377735 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f58480bc0e0 con 0x7f5858101880 2024-05-09T08:30:05.535 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 msgr2=0x7f583c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 0x7f583c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5858075350 tx=0x7f5840005f90 comp rx=0 tx=0).stop 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 msgr2=0x7f58580755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5848031440 tx=0x7f5848032ce0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 shutdown_connections 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f583c0a2c10 0x7f583c0a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5858103430 0x7f5858074160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5858102a70 0x7f5858073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 --2- 172.21.15.133:0/3233377735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5858101880 0x7f58580755e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 >> 172.21.15.133:0/3233377735 conn(0x7f58580fce10 msgr2=0x7f5858105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:05.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 shutdown_connections 2024-05-09T08:30:05.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.538+0000 7f585ee00640 1 -- 172.21.15.133:0/3233377735 wait complete. 2024-05-09T08:30:05.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:30:05.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.760+0000 7f556ee00640 1 -- 172.21.15.133:0/2140603971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55680761f0 msgr2=0x7f5568076610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.760+0000 7f556ee00640 1 --2- 172.21.15.133:0/2140603971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55680761f0 0x7f5568076610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5558009900 tx=0x7f555802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 -- 172.21.15.133:0/2140603971 shutdown_connections 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 --2- 172.21.15.133:0/2140603971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077dd0 0x7f5568078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 --2- 172.21.15.133:0/2140603971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5568077430 0x7f5568077890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 --2- 172.21.15.133:0/2140603971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55680761f0 0x7f5568076610 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 -- 172.21.15.133:0/2140603971 >> 172.21.15.133:0/2140603971 conn(0x7f55680ff020 msgr2=0x7f5568101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 -- 172.21.15.133:0/2140603971 shutdown_connections 2024-05-09T08:30:05.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.761+0000 7f556ee00640 1 -- 172.21.15.133:0/2140603971 wait complete. 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 Processor -- start 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 -- start start 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55680761f0 0x7f556819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 0x7f556819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5568113b80 con 0x7f5568077dd0 2024-05-09T08:30:05.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5568113a00 con 0x7f55680761f0 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5568113d00 con 0x7f5568077430 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48224/0 (socket says 172.21.15.133:48224) 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.762+0000 7f556ce00640 1 -- 172.21.15.133:0/2002757417 learned_addr learned my addr 172.21.15.133:0/2002757417 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 0x7f556819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567e00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55680761f0 0x7f556819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 -- 172.21.15.133:0/2002757417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55680761f0 msgr2=0x7f556819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55680761f0 0x7f556819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 -- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 msgr2=0x7f55681a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 -- 172.21.15.133:0/2002757417 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681a5da0 con 0x7f5568077430 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f556ce00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.763+0000 7f5567400640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 0x7f556819e6c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:05.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.764+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400cd80 con 0x7f5568077430 2024-05-09T08:30:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.764+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681a6090 con 0x7f5568077430 2024-05-09T08:30:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.764+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55540035f0 con 0x7f5568077430 2024-05-09T08:30:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.764+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554010670 con 0x7f5568077430 2024-05-09T08:30:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.764+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a6630 con 0x7f5568077430 2024-05-09T08:30:05.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.765+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568110570 con 0x7f5568077430 2024-05-09T08:30:05.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.768+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f55540037a0 con 0x7f5568077430 2024-05-09T08:30:05.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.769+0000 7f5564c00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 0x7f554c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:05.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.769+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f55540cf350 con 0x7f5568077430 2024-05-09T08:30:05.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.769+0000 7f5567e00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 0x7f554c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:05.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.770+0000 7f5567e00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 0x7f554c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5558031440 tx=0x7f55580315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:05.773+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f555409c280 con 0x7f5568077430 2024-05-09T08:30:06.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.016+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5568062430 con 0x7f5568077430 2024-05-09T08:30:06.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.017+0000 7f5564c00640 1 -- 172.21.15.133:0/2002757417 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7f555409bb80 con 0x7f5568077430 2024-05-09T08:30:06.018 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:30:06.018 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:30:06.018 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 msgr2=0x7f554c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 0x7f554c0a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5558031440 tx=0x7f55580315b0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 msgr2=0x7f556819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 0x7f556819e6c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400eeb0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 shutdown_connections 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f554c0a2b90 0x7f554c0a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5568077dd0 0x7f55681a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5568077430 0x7f556819e6c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 --2- 172.21.15.133:0/2002757417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55680761f0 0x7f556819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 >> 172.21.15.133:0/2002757417 conn(0x7f55680ff020 msgr2=0x7f5568100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 shutdown_connections 2024-05-09T08:30:06.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.020+0000 7f556ee00640 1 -- 172.21.15.133:0/2002757417 wait complete. 2024-05-09T08:30:06.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:30:06.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:05 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1410791836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.099 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:05 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3233377735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:05 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1410791836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:05 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3233377735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:05 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1410791836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:05 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3233377735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:06.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.228+0000 7f2ca3c00640 1 -- 172.21.15.133:0/174876577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077430 msgr2=0x7f2c9c077890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.228+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/174876577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077430 0x7f2c9c077890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009900 tx=0x7f2c8c02a3f0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 -- 172.21.15.133:0/174876577 shutdown_connections 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/174876577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c078670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/174876577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077430 0x7f2c9c077890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/174876577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 -- 172.21.15.133:0/174876577 >> 172.21.15.133:0/174876577 conn(0x7f2c9c0ff020 msgr2=0x7f2c9c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 -- 172.21.15.133:0/174876577 shutdown_connections 2024-05-09T08:30:06.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.229+0000 7f2ca3c00640 1 -- 172.21.15.133:0/174876577 wait complete. 2024-05-09T08:30:06.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 Processor -- start 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 -- start start 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077430 0x7f2c9c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c1a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c113b80 con 0x7f2c9c077dd0 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c113a00 con 0x7f2c9c077430 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.230+0000 7f2ca3c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c113d00 con 0x7f2c9c0761f0 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c1a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:58786/0 (socket says 172.21.15.133:58786) 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 -- 172.21.15.133:0/1187223917 learned_addr learned my addr 172.21.15.133:0/1187223917 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:06.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2c93e00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077430 0x7f2c9c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 -- 172.21.15.133:0/1187223917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077430 msgr2=0x7f2c9c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077430 0x7f2c9c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 -- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 msgr2=0x7f2c9c1a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c1a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 -- 172.21.15.133:0/1187223917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c1a5e20 con 0x7f2c9c0761f0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.231+0000 7f2ca1200640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400e9f0 tx=0x7f2c8400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.232+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400ccc0 con 0x7f2c9c0761f0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.232+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c1a6110 con 0x7f2c9c0761f0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.232+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8400ce60 con 0x7f2c9c0761f0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.232+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84010690 con 0x7f2c9c0761f0 2024-05-09T08:30:06.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.232+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c1a65e0 con 0x7f2c9c0761f0 2024-05-09T08:30:06.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.233+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c110600 con 0x7f2c9c0761f0 2024-05-09T08:30:06.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.235+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2c840046c0 con 0x7f2c9c0761f0 2024-05-09T08:30:06.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.236+0000 7f2c9a000640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.237+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2c840cf430 con 0x7f2c9c0761f0 2024-05-09T08:30:06.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.237+0000 7f2c93e00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.237+0000 7f2c93e00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 0x7f2c800a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c031440 tx=0x7f2c8c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:06.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.241+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c8409c3c0 con 0x7f2c9c0761f0 2024-05-09T08:30:06.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.444+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c9c062430 con 0x7f2c9c0761f0 2024-05-09T08:30:06.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.446+0000 7f2c9a000640 1 -- 172.21.15.133:0/1187223917 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7f2c8409bcc0 con 0x7f2c9c0761f0 2024-05-09T08:30:06.448 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:30:06.448 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:30:06.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.449+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 msgr2=0x7f2c800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.449+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 0x7f2c800a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c031440 tx=0x7f2c8c0315b0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.449+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 msgr2=0x7f2c9c19e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.449+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400e9f0 tx=0x7f2c8400edf0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 shutdown_connections 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f2c800a2b90 0x7f2c800a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c9c077dd0 0x7f2c9c1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c9c077430 0x7f2c9c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 --2- 172.21.15.133:0/1187223917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c9c0761f0 0x7f2c9c19e200 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 >> 172.21.15.133:0/1187223917 conn(0x7f2c9c0ff020 msgr2=0x7f2c9c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 shutdown_connections 2024-05-09T08:30:06.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.450+0000 7f2ca3c00640 1 -- 172.21.15.133:0/1187223917 wait complete. 2024-05-09T08:30:06.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:30:06.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 -- 172.21.15.133:0/696186580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240101740 msgr2=0x7fe240101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 --2- 172.21.15.133:0/696186580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240101740 0x7fe240101b60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe230009900 tx=0x7fe23002a410 comp rx=0 tx=0).stop 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 -- 172.21.15.133:0/696186580 shutdown_connections 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 --2- 172.21.15.133:0/696186580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2401032f0 0x7fe240109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 --2- 172.21.15.133:0/696186580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe240102930 0x7fe240102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 --2- 172.21.15.133:0/696186580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240101740 0x7fe240101b60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 -- 172.21.15.133:0/696186580 >> 172.21.15.133:0/696186580 conn(0x7fe2400fccd0 msgr2=0x7fe2400ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 -- 172.21.15.133:0/696186580 shutdown_connections 2024-05-09T08:30:06.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.671+0000 7fe245200640 1 -- 172.21.15.133:0/696186580 wait complete. 2024-05-09T08:30:06.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.672+0000 7fe245200640 1 Processor -- start 2024-05-09T08:30:06.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 -- start start 2024-05-09T08:30:06.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240102930 0x7fe240073c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 0x7fe240074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010d390 con 0x7fe240101740 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010d210 con 0x7fe240102930 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe245200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe24010d510 con 0x7fe2401032f0 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe23f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe23f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48258/0 (socket says 172.21.15.133:48258) 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe23f400640 1 -- 172.21.15.133:0/715440525 learned_addr learned my addr 172.21.15.133:0/715440525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe23ea00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240102930 0x7fe240073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.673+0000 7fe23fe00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 0x7fe240074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 -- 172.21.15.133:0/715440525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240102930 msgr2=0x7fe240073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240102930 0x7fe240073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 -- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 msgr2=0x7fe2400755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 -- 172.21.15.133:0/715440525 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400749f0 con 0x7fe2401032f0 2024-05-09T08:30:06.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23f400640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe23fe00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 0x7fe240074160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe238009fd0 tx=0x7fe23800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23800ce20 con 0x7fe2401032f0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2401a1b60 con 0x7fe2401032f0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.674+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe238005020 con 0x7fe2401032f0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.675+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238010720 con 0x7fe2401032f0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.675+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2401a2060 con 0x7fe2401032f0 2024-05-09T08:30:06.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.676+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe240108ae0 con 0x7fe2401032f0 2024-05-09T08:30:06.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.678+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe238005540 con 0x7fe2401032f0 2024-05-09T08:30:06.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.679+0000 7fe237e00640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 0x7fe2240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:06.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.679+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe2380cf7a0 con 0x7fe2401032f0 2024-05-09T08:30:06.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.679+0000 7fe23f400640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 0x7fe2240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:06.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.680+0000 7fe23f400640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 0x7fe2240a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe230000c00 tx=0x7fe2300046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.683+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe23809c6d0 con 0x7fe2401032f0 2024-05-09T08:30:06.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.882+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe240062430 con 0x7fe2401032f0 2024-05-09T08:30:06.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.883+0000 7fe237e00640 1 -- 172.21.15.133:0/715440525 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe23809bfd0 con 0x7fe2401032f0 2024-05-09T08:30:06.883 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:30:06.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 msgr2=0x7fe2240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 0x7fe2240a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe230000c00 tx=0x7fe2300046c0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 msgr2=0x7fe240074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 0x7fe240074160 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe238009fd0 tx=0x7fe23800ef10 comp rx=0 tx=0).stop 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 shutdown_connections 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fe2240a2b90 0x7fe2240a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2401032f0 0x7fe240074160 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe240102930 0x7fe240073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 --2- 172.21.15.133:0/715440525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe240101740 0x7fe2400755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 >> 172.21.15.133:0/715440525 conn(0x7fe2400fccd0 msgr2=0x7fe240105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 shutdown_connections 2024-05-09T08:30:06.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:06.886+0000 7fe245200640 1 -- 172.21.15.133:0/715440525 wait complete. 2024-05-09T08:30:06.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 -- 172.21.15.133:0/3278181615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780108f60 msgr2=0x7f97801093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 --2- 172.21.15.133:0/3278181615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780108f60 0x7f97801093e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9768009900 tx=0x7f976802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 -- 172.21.15.133:0/3278181615 shutdown_connections 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 --2- 172.21.15.133:0/3278181615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780109920 0x7f97801101b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 --2- 172.21.15.133:0/3278181615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780108f60 0x7f97801093e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 --2- 172.21.15.133:0/3278181615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f9780108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 -- 172.21.15.133:0/3278181615 >> 172.21.15.133:0/3278181615 conn(0x7f9780076d00 msgr2=0x7f9780079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 -- 172.21.15.133:0/3278181615 shutdown_connections 2024-05-09T08:30:07.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.108+0000 7f9785600640 1 -- 172.21.15.133:0/3278181615 wait complete. 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 Processor -- start 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 -- start start 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f978019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780108f60 0x7f978019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113a40 con 0x7f9780109920 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f97801138c0 con 0x7f9780107d70 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9785600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113bc0 con 0x7f9780108f60 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.109+0000 7f9784c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48288/0 (socket says 172.21.15.133:48288) 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 -- 172.21.15.133:0/2896194179 learned_addr learned my addr 172.21.15.133:0/2896194179 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:07.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9777400640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780108f60 0x7f978019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f977fe00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f978019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 -- 172.21.15.133:0/2896194179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780108f60 msgr2=0x7f978019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780108f60 0x7f978019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 -- 172.21.15.133:0/2896194179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 msgr2=0x7f978019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f978019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 -- 172.21.15.133:0/2896194179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97801a5c30 con 0x7f9780109920 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f977fe00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f978019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9784c00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f977800eb10 tx=0x7f977800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778004b20 con 0x7f9780109920 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9778004cc0 con 0x7f9780109920 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778010760 con 0x7f9780109920 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.110+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97801a5f20 con 0x7f9780109920 2024-05-09T08:30:07.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.111+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a6390 con 0x7f9780109920 2024-05-09T08:30:07.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.112+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978010f030 con 0x7f9780109920 2024-05-09T08:30:07.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.113+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9778005020 con 0x7f9780109920 2024-05-09T08:30:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.115+0000 7f977d600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 0x7f97640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.115+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f97780cf7c0 con 0x7f9780109920 2024-05-09T08:30:07.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.115+0000 7f977fe00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 0x7f97640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.116+0000 7f977fe00640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 0x7f97640a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9780108dc0 tx=0x7f9770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.119+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f977809c6f0 con 0x7f9780109920 2024-05-09T08:30:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:06 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2002757417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:06 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1187223917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:06 smithi160 ceph-mon[34070]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:06 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2002757417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:06 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1187223917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:06 smithi133 ceph-mon[29610]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:06 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2002757417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:06 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1187223917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:06 smithi188 ceph-mon[34645]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:07.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.315+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f978002d400 con 0x7f9780109920 2024-05-09T08:30:07.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.316+0000 7f977d600640 1 -- 172.21.15.133:0/2896194179 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f977809bff0 con 0x7f9780109920 2024-05-09T08:30:07.317 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:30:07.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 msgr2=0x7f97640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 0x7f97640a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9780108dc0 tx=0x7f9770005f90 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 msgr2=0x7f97801a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f977800eb10 tx=0x7f977800c2d0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 shutdown_connections 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f97640a2b90 0x7f97640a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9780109920 0x7f97801a55c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9780108f60 0x7f978019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 --2- 172.21.15.133:0/2896194179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9780107d70 0x7f978019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 >> 172.21.15.133:0/2896194179 conn(0x7f9780076d00 msgr2=0x7f978010c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 shutdown_connections 2024-05-09T08:30:07.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.319+0000 7f9785600640 1 -- 172.21.15.133:0/2896194179 wait complete. 2024-05-09T08:30:07.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:30:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3389986105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 msgr2=0x7fd1b4105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3389986105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b4105bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009860 tx=0x7fd19c02a340 comp rx=0 tx=0).stop 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3389986105 shutdown_connections 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3389986105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4106210 0x7fd1b41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3389986105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b4105bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3389986105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4100ed0 0x7fd1b41032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.536+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3389986105 >> 172.21.15.133:0/3389986105 conn(0x7fd1b40faef0 msgr2=0x7fd1b40fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.537+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3389986105 shutdown_connections 2024-05-09T08:30:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.537+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3389986105 wait complete. 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.538+0000 7fd1b9e00640 1 Processor -- start 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.538+0000 7fd1b9e00640 1 -- start start 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.538+0000 7fd1b9e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4100ed0 0x7fd1b4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.538+0000 7fd1b9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.538+0000 7fd1b9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410c2e0 con 0x7fd1b4106210 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410c160 con 0x7fd1b4100ed0 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b410c460 con 0x7fd1b4103800 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48304/0 (socket says 172.21.15.133:48304) 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1abe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 -- 172.21.15.133:0/3947922715 learned_addr learned my addr 172.21.15.133:0/3947922715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b8a00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4100ed0 0x7fd1b4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 -- 172.21.15.133:0/3947922715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 msgr2=0x7fd1b419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 -- 172.21.15.133:0/3947922715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4100ed0 msgr2=0x7fd1b4199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4100ed0 0x7fd1b4199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 -- 172.21.15.133:0/3947922715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a1910 con 0x7fd1b4106210 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.539+0000 7fd1b9400640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00eb10 tx=0x7fd1ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.540+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00ce20 con 0x7fd1b4106210 2024-05-09T08:30:07.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.540+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ac005020 con 0x7fd1b4106210 2024-05-09T08:30:07.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.540+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a1c00 con 0x7fd1b4106210 2024-05-09T08:30:07.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.540+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac010690 con 0x7fd1b4106210 2024-05-09T08:30:07.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.540+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41a21a0 con 0x7fd1b4106210 2024-05-09T08:30:07.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.541+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b41a2370 con 0x7fd1b4106210 2024-05-09T08:30:07.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.543+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd1ac005540 con 0x7fd1b4106210 2024-05-09T08:30:07.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.545+0000 7fd1b2000640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 0x7fd1980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.545+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1ac0cf770 con 0x7fd1b4106210 2024-05-09T08:30:07.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.545+0000 7fd1b8a00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 0x7fd1980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.546+0000 7fd1b8a00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 0x7fd1980a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1b419b1d0 tx=0x7fd1a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.549+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1ac09c6a0 con 0x7fd1b4106210 2024-05-09T08:30:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.749+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1b41a2540 con 0x7fd1b4106210 2024-05-09T08:30:07.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.750+0000 7fd1b2000640 1 -- 172.21.15.133:0/3947922715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fd1ac09bfa0 con 0x7fd1b4106210 2024-05-09T08:30:07.751 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 msgr2=0x7fd1980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 0x7fd1980a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1b419b1d0 tx=0x7fd1a4009260 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 msgr2=0x7fd1b41a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00eb10 tx=0x7fd1ac00ef10 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 shutdown_connections 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1980a2c10 0x7fd1980a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1b4106210 0x7fd1b41a1270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1b4103800 0x7fd1b419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 --2- 172.21.15.133:0/3947922715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1b4100ed0 0x7fd1b4199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 >> 172.21.15.133:0/3947922715 conn(0x7fd1b40faef0 msgr2=0x7fd1b41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 shutdown_connections 2024-05-09T08:30:07.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.753+0000 7fd1b9e00640 1 -- 172.21.15.133:0/3947922715 wait complete. 2024-05-09T08:30:07.773 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:30:07.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 -- 172.21.15.133:0/3089946711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 msgr2=0x7f2448101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 --2- 172.21.15.133:0/3089946711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448101ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2438009860 tx=0x7f243802a340 comp rx=0 tx=0).stop 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 -- 172.21.15.133:0/3089946711 shutdown_connections 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 --2- 172.21.15.133:0/3089946711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448103430 0x7f2448109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 --2- 172.21.15.133:0/3089946711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448102a70 0x7f2448102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 --2- 172.21.15.133:0/3089946711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448101ca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 -- 172.21.15.133:0/3089946711 >> 172.21.15.133:0/3089946711 conn(0x7f24480fce30 msgr2=0x7f24480ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 -- 172.21.15.133:0/3089946711 shutdown_connections 2024-05-09T08:30:07.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.975+0000 7f244d000640 1 -- 172.21.15.133:0/3089946711 wait complete. 2024-05-09T08:30:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 Processor -- start 2024-05-09T08:30:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 -- start start 2024-05-09T08:30:07.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448102a70 0x7f244819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448103430 0x7f24481a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d500 con 0x7f2448103430 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d380 con 0x7f2448102a70 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.976+0000 7f244d000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f244810d680 con 0x7f2448101880 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2446800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448103430 0x7f24481a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:58868/0 (socket says 172.21.15.133:58868) 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 -- 172.21.15.133:0/232394101 learned_addr learned my addr 172.21.15.133:0/232394101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445400640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448102a70 0x7f244819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 -- 172.21.15.133:0/232394101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448102a70 msgr2=0x7f244819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448102a70 0x7f244819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 -- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448103430 msgr2=0x7f24481a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:07.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448103430 0x7f24481a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 -- 172.21.15.133:0/232394101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24481a19e0 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f2445e00640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24380305b0 tx=0x7f2438033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.977+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243803f020 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.978+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438030850 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.978+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24481a1c70 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.978+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438037040 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.978+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481a2160 con 0x7f2448101880 2024-05-09T08:30:07.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.979+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24481a2330 con 0x7f2448101880 2024-05-09T08:30:07.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.980+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2438042020 con 0x7f2448101880 2024-05-09T08:30:07.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.982+0000 7f243ea00640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 0x7f242c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:07.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.982+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f24380ef390 con 0x7f2448101880 2024-05-09T08:30:07.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.982+0000 7f2445400640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 0x7f242c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:07.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.983+0000 7f2445400640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 0x7f242c0a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f244819b2d0 tx=0x7f2434009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:07.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:07.986+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2438049020 con 0x7f2448101880 2024-05-09T08:30:08.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:07 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/715440525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:07 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2896194179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:07 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3947922715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:07 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/715440525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:07 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2896194179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:07 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3947922715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:07 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/715440525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:07 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2896194179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:07 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3947922715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:30:08.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.228+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24481a2500 con 0x7f2448101880 2024-05-09T08:30:08.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.230+0000 7f243ea00640 1 -- 172.21.15.133:0/232394101 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11888 (secure 0 0 0) 0x7f24380bc2c0 con 0x7f2448101880 2024-05-09T08:30:08.230 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:30:08.230 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:28:01.270664+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:30:08.231 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 22 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 msgr2=0x7f242c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 0x7f242c0a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f244819b2d0 tx=0x7f2434009260 comp rx=0 tx=0).stop 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 msgr2=0x7f2448199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24380305b0 tx=0x7f2438033d90 comp rx=0 tx=0).stop 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 shutdown_connections 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f242c0a2b90 0x7f242c0a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2448103430 0x7f24481a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2448102a70 0x7f244819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 --2- 172.21.15.133:0/232394101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2448101880 0x7f2448199dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.232+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 >> 172.21.15.133:0/232394101 conn(0x7f24480fce30 msgr2=0x7f2448073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.233+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 shutdown_connections 2024-05-09T08:30:08.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.233+0000 7f244d000640 1 -- 172.21.15.133:0/232394101 wait complete. 2024-05-09T08:30:08.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:30:08.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.437+0000 7f8f7b200640 1 -- 172.21.15.133:0/2066882236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 msgr2=0x7f8f7c1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.437+0000 7f8f7b200640 1 --2- 172.21.15.133:0/2066882236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c1080d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009860 tx=0x7f8f6c02a340 comp rx=0 tx=0).stop 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 -- 172.21.15.133:0/2066882236 shutdown_connections 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 --2- 172.21.15.133:0/2066882236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 --2- 172.21.15.133:0/2066882236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 0x7f8f7c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 --2- 172.21.15.133:0/2066882236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c1080d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 -- 172.21.15.133:0/2066882236 >> 172.21.15.133:0/2066882236 conn(0x7f8f7c076c70 msgr2=0x7f8f7c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 -- 172.21.15.133:0/2066882236 shutdown_connections 2024-05-09T08:30:08.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.438+0000 7f8f7b200640 1 -- 172.21.15.133:0/2066882236 wait complete. 2024-05-09T08:30:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 Processor -- start 2024-05-09T08:30:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 -- start start 2024-05-09T08:30:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 0x7f8f7c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c113920 con 0x7f8f7c109860 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1137a0 con 0x7f8f7c108ea0 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c113aa0 con 0x7f8f7c107cb0 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48344/0 (socket says 172.21.15.133:48344) 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f7a800640 1 -- 172.21.15.133:0/630278283 learned_addr learned my addr 172.21.15.133:0/630278283 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f71400640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 0x7f8f7c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.439+0000 7f8f79e00640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 -- 172.21.15.133:0/630278283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 msgr2=0x7f8f7c19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 -- 172.21.15.133:0/630278283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 msgr2=0x7f8f7c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 0x7f8f7c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 -- 172.21.15.133:0/630278283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1a5bf0 con 0x7f8f7c109860 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7a800640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f7400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74004b20 con 0x7f8f7c109860 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f74003e50 con 0x7f8f7c109860 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c1a5ee0 con 0x7f8f7c109860 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400b630 con 0x7f8f7c109860 2024-05-09T08:30:08.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.440+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a6350 con 0x7f8f7c109860 2024-05-09T08:30:08.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.442+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c10f0a0 con 0x7f8f7c109860 2024-05-09T08:30:08.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.444+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8f7401d020 con 0x7f8f7c109860 2024-05-09T08:30:08.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.445+0000 7f8f73400640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 0x7f8f600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.445+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8f740cefe0 con 0x7f8f7c109860 2024-05-09T08:30:08.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.445+0000 7f8f79e00640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 0x7f8f600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.446+0000 7f8f79e00640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 0x7f8f600a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009830 tx=0x7f8f6c030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.450+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f7409bf50 con 0x7f8f7c109860 2024-05-09T08:30:08.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.650+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f7c0624d0 con 0x7f8f7c109860 2024-05-09T08:30:08.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.654+0000 7f8f73400640 1 -- 172.21.15.133:0/630278283 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7f8f7409b850 con 0x7f8f7c109860 2024-05-09T08:30:08.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:30:08.655 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:30:08.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 msgr2=0x7f8f600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 0x7f8f600a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c009830 tx=0x7f8f6c030570 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 msgr2=0x7f8f7c1a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009fd0 tx=0x7f8f7400c800 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 shutdown_connections 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8f600a2b90 0x7f8f600a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f7c109860 0x7f8f7c1a5580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7c108ea0 0x7f8f7c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 --2- 172.21.15.133:0/630278283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f7c107cb0 0x7f8f7c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 >> 172.21.15.133:0/630278283 conn(0x7f8f7c076c70 msgr2=0x7f8f7c10c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.657+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 shutdown_connections 2024-05-09T08:30:08.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.658+0000 7f8f7b200640 1 -- 172.21.15.133:0/630278283 wait complete. 2024-05-09T08:30:08.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T08:27:55.241118+0000', 'modified': '2024-05-09T08:28:01.270664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24566, 'mds_1': 24556, 'mds_2': 24536}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6837/3112088995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 3112088995}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 3112088995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6837/1342622403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 1342622403}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 1342622403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6839/2528648460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 2528648460}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 2528648460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/3807326844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 3807326844}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 3807326844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24556': {'gid': 24556, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.133:6841/2374052560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6840', 'nonce': 2374052560}, {'type': 'v1', 'addr': '172.21.15.133:6841', 'nonce': 2374052560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6839/145157679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 145157679}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 145157679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24536, 24556, 24566]} 2024-05-09T08:30:08.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T08:30:08.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 -- 172.21.15.133:0/3176286039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170104c80 msgr2=0x7fd170105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 --2- 172.21.15.133:0/3176286039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170104c80 0x7fd170105100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd158009860 tx=0x7fd15802a340 comp rx=0 tx=0).stop 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 -- 172.21.15.133:0/3176286039 shutdown_connections 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 --2- 172.21.15.133:0/3176286039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170105640 0x7fd17010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 --2- 172.21.15.133:0/3176286039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170104c80 0x7fd170105100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 --2- 172.21.15.133:0/3176286039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170103a90 0x7fd170103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 -- 172.21.15.133:0/3176286039 >> 172.21.15.133:0/3176286039 conn(0x7fd1700ff020 msgr2=0x7fd170101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 -- 172.21.15.133:0/3176286039 shutdown_connections 2024-05-09T08:30:08.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.866+0000 7fd176c00640 1 -- 172.21.15.133:0/3176286039 wait complete. 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.867+0000 7fd176c00640 1 Processor -- start 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.867+0000 7fd176c00640 1 -- start start 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170104c80 0x7fd17019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 0x7fd1701a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010f6f0 con 0x7fd170104c80 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010f570 con 0x7fd170103a90 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd176c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd17010f870 con 0x7fd170105640 2024-05-09T08:30:08.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37272/0 (socket says 172.21.15.133:37272) 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd167400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170104c80 0x7fd17019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 -- 172.21.15.133:0/700353811 learned_addr learned my addr 172.21.15.133:0/700353811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd174c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 0x7fd1701a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 -- 172.21.15.133:0/700353811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 msgr2=0x7fd1701a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 0x7fd1701a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 -- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170104c80 msgr2=0x7fd17019a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170104c80 0x7fd17019a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd16fe00640 1 -- 172.21.15.133:0/700353811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1701a1a10 con 0x7fd170103a90 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.868+0000 7fd174c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 0x7fd1701a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd16fe00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e9c0 tx=0x7fd16000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd16d600640 1 -- 172.21.15.133:0/700353811 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000cd80 con 0x7fd170103a90 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd16d600640 1 -- 172.21.15.133:0/700353811 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1600035f0 con 0x7fd170103a90 2024-05-09T08:30:08.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1701a1d00 con 0x7fd170103a90 2024-05-09T08:30:08.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd16d600640 1 -- 172.21.15.133:0/700353811 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160010690 con 0x7fd170103a90 2024-05-09T08:30:08.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.869+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1701a21d0 con 0x7fd170103a90 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd16d600640 1 -- 172.21.15.133:0/700353811 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd1600037a0 con 0x7fd170103a90 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 Processor -- start 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 -- start start 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2eed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2eeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584d2e4b480 con 0x5584d2ee89a0 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5584d2df35c0 con 0x5584d2ee7fb0 2024-05-09T08:30:08.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.872+0000 7fd179ba2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584d2e47440 con 0x5584d2eedb10 2024-05-09T08:30:08.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2eed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd174c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2eed4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48376/0 (socket says 172.21.15.133:48376) 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd174c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37276/0 (socket says 172.21.15.133:37276) 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2eeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd16fe00640 1 -- 172.21.15.133:0/4224995370 learned_addr learned my addr 172.21.15.133:0/4224995370 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 -- 172.21.15.133:0/4224995370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 msgr2=0x5584d2ee83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 -- 172.21.15.133:0/4224995370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 msgr2=0x5584d2eed4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2eed4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 -- 172.21.15.133:0/4224995370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d2b5f2d0 con 0x5584d2eedb10 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd174c00640 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.873+0000 7fd167400640 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2eeffa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd158035040 tx=0x7fd1580095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e435e2018a3e588 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd165200640 1 -- 172.21.15.133:0/4224995370 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158037040 con 0x5584d2eedb10 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd165200640 1 -- 172.21.15.133:0/4224995370 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158042070 con 0x5584d2eedb10 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd165200640 1 -- 172.21.15.133:0/4224995370 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158005380 con 0x5584d2eedb10 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd16d600640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 0x7fd1540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd174c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 0x7fd1540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd16d600640 1 -- 172.21.15.133:0/700353811 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1600cf4f0 con 0x7fd170103a90 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd179ba2740 1 -- 172.21.15.133:0/4224995370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 msgr2=0x5584d2eeffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.874+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2eeffa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd158035040 tx=0x7fd1580095a0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd174c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 0x7fd1540a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5584d2ee9200 tx=0x7fd168011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 -- 172.21.15.133:0/4224995370 shutdown_connections 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2eeffa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2eed4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4224995370 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ee83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 -- 172.21.15.133:0/4224995370 >> 172.21.15.133:0/4224995370 conn(0x5584d2eaf8a0 msgr2=0x5584d2ee3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 -- 172.21.15.133:0/4224995370 shutdown_connections 2024-05-09T08:30:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 -- 172.21.15.133:0/4224995370 wait complete. 2024-05-09T08:30:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 -- start start 2024-05-09T08:30:08.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.875+0000 7fd179ba2740 1 Processor -- start 2024-05-09T08:30:08.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee89a0 0x5584d2ef2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584d2ef0a60 con 0x5584d2ee7fb0 2024-05-09T08:30:08.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5584d2e06980 con 0x5584d2eedb10 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584d2ef0be0 con 0x5584d2ee89a0 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd179ba2740 10 client.0 _ll_register_callbacks cb 0x5584d2e41fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd167400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee89a0 0x5584d2ef2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48392/0 (socket says 172.21.15.133:48392) 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 -- 172.21.15.133:0/1571289023 learned_addr learned my addr 172.21.15.133:0/1571289023 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd174c00640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 -- 172.21.15.133:0/1571289023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee89a0 msgr2=0x5584d2ef2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee89a0 0x5584d2ef2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 -- 172.21.15.133:0/1571289023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 msgr2=0x5584d2ef9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.876+0000 7fd16fe00640 1 -- 172.21.15.133:0/1571289023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d2c296b0 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd1600a4460 tx=0x7fd1600ccf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d76fd34e7f8b6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16009b860 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1600cf0d0 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd158042070 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd16009b680 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd179ba2740 10 client.15036 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584d2efc600 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158042070 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd16009b170 con 0x5584d2ee7fb0 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.877+0000 7fd179ba2740 10 client.15036 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd179ba2740 10 client.15036 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd179ba2740 20 client.15036 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2efeca0 0x5584d2f00110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd179ba2740 4 client.15036 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 --> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d2e5f060 con 0x5584d2efeca0 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd167400640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2efeca0 0x5584d2f00110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd167400640 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2efeca0 0x5584d2f00110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48abb681e00898b8 server_cookie=f8d5d307d8cfe9b1 in_seq=0 out_seq=0 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.878+0000 7fd15d600640 10 client.15036 ms_handle_connect on v2:172.21.15.133:6838/3807326844 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd15d600640 1 -- 172.21.15.133:0/1571289023 <== mds.2 v2:172.21.15.133:6838/3807326844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584d2e5f060 con 0x5584d2efeca0 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd15d600640 10 client.15036 handle_command_reply: tid=0 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 client.15036 shutdown 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 msgr2=0x5584d2ef1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd1600a4460 tx=0x7fd1600ccf80 comp rx=0 tx=0).stop 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 shutdown_connections 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2efeca0 0x5584d2f00110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee89a0 0x5584d2ef2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1571289023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef1fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd15d600640 0 client.15036 ms_handle_reset on v2:172.21.15.133:6838/3807326844 2024-05-09T08:30:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.879+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 >> 172.21.15.133:0/1571289023 conn(0x5584d2eaf8a0 msgr2=0x5584d2ee2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.880+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 shutdown_connections 2024-05-09T08:30:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.880+0000 7fd179ba2740 1 -- 172.21.15.133:0/1571289023 wait complete. 2024-05-09T08:30:08.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.880+0000 7fd179ba2740 20 client.15036 trim_cache size 0 max 16384 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 Processor -- start 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 -- start start 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee7fb0 0x5584d2eeff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584d2efe450 con 0x5584d2ee89a0 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5584d2efeb20 con 0x5584d2eedb10 2024-05-09T08:30:08.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.899+0000 7fd179ba2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584d2eff1f0 con 0x5584d2ee7fb0 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37298/0 (socket says 172.21.15.133:37298) 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 -- 172.21.15.133:0/4191462971 learned_addr learned my addr 172.21.15.133:0/4191462971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd167400640 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd16fe00640 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee7fb0 0x5584d2eeff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 -- 172.21.15.133:0/4191462971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee7fb0 msgr2=0x5584d2eeff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee7fb0 0x5584d2eeff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 -- 172.21.15.133:0/4191462971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 msgr2=0x5584d2ef9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 -- 172.21.15.133:0/4191462971 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d2ee8d80 con 0x5584d2eedb10 2024-05-09T08:30:08.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.900+0000 7fd174c00640 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd16800f370 tx=0x7fd16801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b71b7341fa7654 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd15fe00640 1 -- 172.21.15.133:0/4191462971 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16801b820 con 0x5584d2eedb10 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd15fe00640 1 -- 172.21.15.133:0/4191462971 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16802c070 con 0x5584d2eedb10 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd15fe00640 1 -- 172.21.15.133:0/4191462971 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168027d20 con 0x5584d2eedb10 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 -- 172.21.15.133:0/4191462971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 msgr2=0x5584d2ef2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd16800f370 tx=0x7fd16801a000 comp rx=0 tx=0).stop 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 -- 172.21.15.133:0/4191462971 shutdown_connections 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2eedb10 0x5584d2ef2300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 --2- 172.21.15.133:0/4191462971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2ee7fb0 0x5584d2eeff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 -- 172.21.15.133:0/4191462971 >> 172.21.15.133:0/4191462971 conn(0x5584d2eaf8a0 msgr2=0x5584d2ef1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 -- 172.21.15.133:0/4191462971 shutdown_connections 2024-05-09T08:30:08.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.901+0000 7fd179ba2740 1 -- 172.21.15.133:0/4191462971 wait complete. 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 -- start start 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 Processor -- start 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2f62c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584d2efff90 con 0x5584d2ee89a0 2024-05-09T08:30:08.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x5584d2eff8c0 con 0x5584d2ee7fb0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584d2f00660 con 0x5584d2eedb10 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd179ba2740 10 client.0 _ll_register_callbacks cb 0x7fd1600046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.902+0000 7fd167400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48398/0 (socket says 172.21.15.133:48398) 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 -- 172.21.15.133:0/1392825517 learned_addr learned my addr 172.21.15.133:0/1392825517 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd174c00640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2f62c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 -- 172.21.15.133:0/1392825517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 msgr2=0x5584d2f62c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2f62c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 -- 172.21.15.133:0/1392825517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 msgr2=0x5584d2ef4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 -- 172.21.15.133:0/1392825517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d2f631b0 con 0x5584d2ee89a0 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:08.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd167400640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd158009860 tx=0x7fd158005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41c6ff7dd87afc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd15d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158055060 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd15804fbc0 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.903+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16802c070 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd158037040 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 10 client.15042 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5584d2f644b0 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16802c070 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd1580052a0 con 0x5584d2ee89a0 2024-05-09T08:30:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 10 client.15042 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 10 client.15042 resolve_mds: resolved name 'g' to daemon mds.g 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 20 client.15042 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2f68c40 0x5584d2f67950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 4 client.15042 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.904+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 --> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] -- command(tid 0: {"prefix": "status"}) -- 0x5584d2e58bb0 con 0x5584d2f68c40 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2f68c40 0x5584d2f67950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd16fe00640 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2f68c40 0x5584d2f67950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2057b1afb946cfb4 server_cookie=41b8e8739e6795e6 in_seq=0 out_seq=0 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd15d600640 10 client.15042 ms_handle_connect on v2:172.21.15.133:6838/3807326844 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd15d600640 1 -- 172.21.15.133:0/1392825517 <== mds.2 v2:172.21.15.133:6838/3807326844 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5584d2e58bb0 con 0x5584d2f68c40 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd15d600640 10 client.15042 handle_command_reply: tid=0 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.905+0000 7fd179ba2740 1 client.15042 shutdown 2024-05-09T08:30:08.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 msgr2=0x5584d2ef49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd158009860 tx=0x7fd158005d90 comp rx=0 tx=0).stop 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 shutdown_connections 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:6838/3807326844,v1:172.21.15.133:6839/3807326844] conn(0x5584d2f68c40 0x5584d2f67950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584d2eedb10 0x5584d2f62c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584d2ee89a0 0x5584d2ef49b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 --2- 172.21.15.133:0/1392825517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5584d2ee7fb0 0x5584d2ef4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd15d600640 0 client.15042 ms_handle_reset on v2:172.21.15.133:6838/3807326844 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 >> 172.21.15.133:0/1392825517 conn(0x5584d2eaf8a0 msgr2=0x5584d2ee6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 shutdown_connections 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 1 -- 172.21.15.133:0/1392825517 wait complete. 2024-05-09T08:30:08.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd179ba2740 20 client.15042 trim_cache size 0 max 16384 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24536, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 128.650735594, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:30:08.912 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 146.860013103 2024-05-09T08:30:08.913 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:30:08.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 msgr2=0x7fd1540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 0x7fd1540a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5584d2ee9200 tx=0x7fd168011040 comp rx=0 tx=0).stop 2024-05-09T08:30:08.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 msgr2=0x7fd170199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.906+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd16000e9c0 tx=0x7fd16000ee80 comp rx=0 tx=0).stop 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 shutdown_connections 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1540a2b90 0x7fd1540a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd170105640 0x7fd1701a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd170104c80 0x7fd17019a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 --2- 172.21.15.133:0/700353811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd170103a90 0x7fd170199df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 >> 172.21.15.133:0/700353811 conn(0x7fd1700ff020 msgr2=0x7fd170108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 shutdown_connections 2024-05-09T08:30:08.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:08.907+0000 7fd176c00640 1 -- 172.21.15.133:0/700353811 wait complete. 2024-05-09T08:30:08.923 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T08:30:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:08 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/232394101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:08 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/630278283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:08.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:08 smithi133 ceph-mon[29610]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:09.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 -- 172.21.15.133:0/3292722234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8104ca0 msgr2=0x7fb4a8105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 --2- 172.21.15.133:0/3292722234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8104ca0 0x7fb4a8105120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb490009ca0 tx=0x7fb49002a8c0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 -- 172.21.15.133:0/3292722234 shutdown_connections 2024-05-09T08:30:09.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 --2- 172.21.15.133:0/3292722234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8105660 0x7fb4a810bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 --2- 172.21.15.133:0/3292722234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8104ca0 0x7fb4a8105120 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 --2- 172.21.15.133:0/3292722234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8103ab0 0x7fb4a8103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 -- 172.21.15.133:0/3292722234 >> 172.21.15.133:0/3292722234 conn(0x7fb4a80ff020 msgr2=0x7fb4a8101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 -- 172.21.15.133:0/3292722234 shutdown_connections 2024-05-09T08:30:09.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.111+0000 7fb4afa00640 1 -- 172.21.15.133:0/3292722234 wait complete. 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 Processor -- start 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 -- start start 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8103ab0 0x7fb4a8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 0x7fb4a819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f760 con 0x7fb4a8104ca0 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f5e0 con 0x7fb4a8105660 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4afa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f8e0 con 0x7fb4a8103ab0 2024-05-09T08:30:09.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.113+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 0x7fb4a819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37320/0 (socket says 172.21.15.133:37320) 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 -- 172.21.15.133:0/19561316 learned_addr learned my addr 172.21.15.133:0/19561316 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 0x7fb4a819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48412/0 (socket says 172.21.15.133:48412) 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ad000640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8103ab0 0x7fb4a8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 -- 172.21.15.133:0/19561316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8103ab0 msgr2=0x7fb4a8199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8103ab0 0x7fb4a8199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 -- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 msgr2=0x7fb4a819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 0x7fb4a819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 -- 172.21.15.133:0/19561316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a19d0 con 0x7fb4a8105660 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4ada00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e970 tx=0x7fb49800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.114+0000 7fb4a5600640 1 -- 172.21.15.133:0/19561316 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800ccc0 con 0x7fb4a8105660 2024-05-09T08:30:09.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.115+0000 7fb4a5600640 1 -- 172.21.15.133:0/19561316 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49800ce60 con 0x7fb4a8105660 2024-05-09T08:30:09.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.115+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a1cc0 con 0x7fb4a8105660 2024-05-09T08:30:09.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.115+0000 7fb4a5600640 1 -- 172.21.15.133:0/19561316 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498010490 con 0x7fb4a8105660 2024-05-09T08:30:09.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.115+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a2190 con 0x7fb4a8105660 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4a5600640 1 -- 172.21.15.133:0/19561316 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb498005020 con 0x7fb4a8105660 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 Processor -- start 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 -- start start 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be65773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560be64da480 con 0x560be65779a0 2024-05-09T08:30:09.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560be64825c0 con 0x560be6576fb0 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560be64d6440 con 0x560be657cb10 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.118+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be65773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48422/0 (socket says 172.21.15.133:48422) 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 -- 172.21.15.133:0/944359847 learned_addr learned my addr 172.21.15.133:0/944359847 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 -- 172.21.15.133:0/944359847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 msgr2=0x560be657efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 -- 172.21.15.133:0/944359847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 msgr2=0x560be65773d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be65773d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 -- 172.21.15.133:0/944359847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be61ee2d0 con 0x560be65779a0 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:09.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4ad000640 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ca0 tx=0x7fb4a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277f604c191965c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb4a5600640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 0x7fb48c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.119+0000 7fb49e000640 1 -- 172.21.15.133:0/944359847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004220 con 0x560be65779a0 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4a5600640 1 -- 172.21.15.133:0/19561316 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb4980cf4e0 con 0x7fb4a8105660 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb49e000640 1 -- 172.21.15.133:0/944359847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a00043c0 con 0x560be65779a0 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb49e000640 1 -- 172.21.15.133:0/944359847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0019930 con 0x560be65779a0 2024-05-09T08:30:09.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 0x7fb48c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 0x7fb48c0a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb490005e30 tx=0x7fb49002a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 -- 172.21.15.133:0/944359847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 msgr2=0x560be657c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ca0 tx=0x7fb4a0011810 comp rx=0 tx=0).stop 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 -- 172.21.15.133:0/944359847 shutdown_connections 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be657c4a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/944359847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be65773d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.120+0000 7fb4b29c9740 1 -- 172.21.15.133:0/944359847 >> 172.21.15.133:0/944359847 conn(0x560be653e8a0 msgr2=0x560be6572a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 -- 172.21.15.133:0/944359847 shutdown_connections 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 -- 172.21.15.133:0/944359847 wait complete. 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 -- start start 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 Processor -- start 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6581040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.121+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 0x560be65885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560be657fac0 con 0x560be657cb10 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560be6495980 con 0x560be6576fb0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560be657fc40 con 0x560be65779a0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4b29c9740 10 client.0 _ll_register_callbacks cb 0x560be64d0fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4ada00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 0x560be65885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:59006/0 (socket says 172.21.15.133:59006) 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 -- 172.21.15.133:0/4089741018 learned_addr learned my addr 172.21.15.133:0/4089741018 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4ad000640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6581040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 -- 172.21.15.133:0/4089741018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 msgr2=0x560be6581040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6581040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 -- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 msgr2=0x560be65885c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 0x560be65885c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4a7e00640 1 -- 172.21.15.133:0/4089741018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be62b86b0 con 0x560be65779a0 2024-05-09T08:30:09.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.122+0000 7fb4ada00640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 0x560be65885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb490035000 tx=0x7fb4900095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c16cc1ba90b6db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb496a00640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490004250 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb4900043f0 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a00043c0 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb4900045f0 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb4b29c9740 10 client.25147 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.123+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560be658b5a0 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a00043c0 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb490053070 con 0x560be65779a0 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 10 client.25147 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 10 client.25147 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 20 client.25147 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:09.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be658dd40 0x560be658f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 4 client.25147 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 --> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be64ee060 con 0x560be658dd40 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.124+0000 7fb4ad000640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be658dd40 0x560be658f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.125+0000 7fb4ad000640 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be658dd40 0x560be658f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49f8c8b1b09e2a48 server_cookie=e23ecd2189232a64 in_seq=0 out_seq=0 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.125+0000 7fb496a00640 10 client.25147 ms_handle_connect on v2:172.21.15.133:6840/2374052560 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.125+0000 7fb496a00640 1 -- 172.21.15.133:0/4089741018 <== mds.1 v2:172.21.15.133:6840/2374052560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560be64ee060 con 0x560be658dd40 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.125+0000 7fb496a00640 10 client.25147 handle_command_reply: tid=0 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.125+0000 7fb4b29c9740 1 client.25147 shutdown 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 msgr2=0x560be6581580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb490035000 tx=0x7fb4900095c0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 shutdown_connections 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be658dd40 0x560be658f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be657cb10 0x560be65885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6581580 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb496a00640 0 client.25147 ms_handle_reset on v2:172.21.15.133:6840/2374052560 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/4089741018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6581040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 >> 172.21.15.133:0/4089741018 conn(0x560be653e8a0 msgr2=0x560be6571b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 shutdown_connections 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 1 -- 172.21.15.133:0/4089741018 wait complete. 2024-05-09T08:30:09.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.126+0000 7fb4b29c9740 20 client.25147 trim_cache size 0 max 16384 2024-05-09T08:30:09.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 Processor -- start 2024-05-09T08:30:09.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 -- start start 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be6588580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560be658d4f0 con 0x560be65779a0 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560be658dbc0 con 0x560be6576fb0 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560be658e290 con 0x560be657cb10 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.146+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:37340/0 (socket says 172.21.15.133:37340) 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 -- 172.21.15.133:0/787733161 learned_addr learned my addr 172.21.15.133:0/787733161 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ada00640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be6588580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 -- 172.21.15.133:0/787733161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 msgr2=0x560be657c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 -- 172.21.15.133:0/787733161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 msgr2=0x560be6588580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be6588580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 -- 172.21.15.133:0/787733161 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be65736e0 con 0x560be6576fb0 2024-05-09T08:30:09.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be6588580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:30:09.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb4ad000640 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbad09356bbe6b60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb49d600640 1 -- 172.21.15.133:0/787733161 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a002bce0 con 0x560be6576fb0 2024-05-09T08:30:09.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb49d600640 1 -- 172.21.15.133:0/787733161 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a0005520 con 0x560be6576fb0 2024-05-09T08:30:09.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.147+0000 7fb49d600640 1 -- 172.21.15.133:0/787733161 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a002b790 con 0x560be6576fb0 2024-05-09T08:30:09.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 -- 172.21.15.133:0/787733161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 msgr2=0x560be6580a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0009310 comp rx=0 tx=0).stop 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 -- 172.21.15.133:0/787733161 shutdown_connections 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be657cb10 0x560be657c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be65779a0 0x560be6588580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/787733161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be6576fb0 0x560be6580a20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 -- 172.21.15.133:0/787733161 >> 172.21.15.133:0/787733161 conn(0x560be653e8a0 msgr2=0x560be6580310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 -- 172.21.15.133:0/787733161 shutdown_connections 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.148+0000 7fb4b29c9740 1 -- 172.21.15.133:0/787733161 wait complete. 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 -- start start 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 Processor -- start 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6584130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 0x560be6587150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560be658f030 con 0x560be6576fb0 2024-05-09T08:30:09.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560be658e960 con 0x560be657cb10 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560be658f700 con 0x560be65779a0 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.149+0000 7fb4b29c9740 10 client.0 _ll_register_callbacks cb 0x7fb490009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 0x560be6587150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ad000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48436/0 (socket says 172.21.15.133:48436) 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ad000640 1 -- 172.21.15.133:0/832117813 learned_addr learned my addr 172.21.15.133:0/832117813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4a7e00640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6584130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 -- 172.21.15.133:0/832117813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 msgr2=0x560be6584130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6584130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 -- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 msgr2=0x560be6583bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 -- 172.21.15.133:0/832117813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be6587740 con 0x560be657cb10 2024-05-09T08:30:09.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ad000640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb4ada00640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 0x560be6587150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb498004fd0 tx=0x7fb49809c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc60e3c8889b6e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb496a00640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.150+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980b8070 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a0005520 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mon.2 v2:172.21.15.160:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb49809bf70 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb498081070 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 10 client.25160 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560be65f3cf0 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mon.2 v2:172.21.15.160:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a0005520 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mon.2 v2:172.21.15.160:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fb4980a45f0 con 0x560be657cb10 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 10 client.25160 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 10 client.25160 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 20 client.25160 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:09.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be65f8830 0x560be65f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 4 client.25160 mds_command: new command op to 24556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 --> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] -- command(tid 0: {"prefix": "status"}) -- 0x560be64e7bb0 con 0x560be65f8830 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.151+0000 7fb4ad000640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be65f8830 0x560be65f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4ad000640 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be65f8830 0x560be65f4c70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8302c8f24ba43de1 server_cookie=2ae6db3988ea7a6d in_seq=0 out_seq=0 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb496a00640 10 client.25160 ms_handle_connect on v2:172.21.15.133:6840/2374052560 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb496a00640 1 -- 172.21.15.133:0/832117813 <== mds.1 v2:172.21.15.133:6840/2374052560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560be64e7bb0 con 0x560be65f8830 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb496a00640 10 client.25160 handle_command_reply: tid=0 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4b29c9740 1 client.25160 shutdown 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 msgr2=0x560be6587150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 0x560be6587150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb498004fd0 tx=0x7fb49809c550 comp rx=0 tx=0).stop 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 shutdown_connections 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.152+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:6840/2374052560,v1:172.21.15.133:6841/2374052560] conn(0x560be65f8830 0x560be65f4c70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560be657cb10 0x560be6587150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560be65779a0 0x560be6584130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 --2- 172.21.15.133:0/832117813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560be6576fb0 0x560be6583bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb496a00640 0 client.25160 ms_handle_reset on v2:172.21.15.133:6840/2374052560 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 >> 172.21.15.133:0/832117813 conn(0x560be653e8a0 msgr2=0x560be6578440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 shutdown_connections 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 1 -- 172.21.15.133:0/832117813 wait complete. 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4b29c9740 20 client.25160 trim_cache size 0 max 16384 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24556, 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:30:09.158 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 129.90754502600001, 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 142.995988963 2024-05-09T08:30:09.159 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 msgr2=0x7fb48c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 0x7fb48c0a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb490005e30 tx=0x7fb49002a8c0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 msgr2=0x7fb4a81a1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e970 tx=0x7fb49800ed70 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 shutdown_connections 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fb48c0a2b90 0x7fb48c0a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4a8105660 0x7fb4a81a1360 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4a8104ca0 0x7fb4a819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 --2- 172.21.15.133:0/19561316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4a8103ab0 0x7fb4a8199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.153+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 >> 172.21.15.133:0/19561316 conn(0x7fb4a80ff020 msgr2=0x7fb4a8107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.154+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 shutdown_connections 2024-05-09T08:30:09.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.154+0000 7fb4afa00640 1 -- 172.21.15.133:0/19561316 wait complete. 2024-05-09T08:30:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:08 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/232394101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:08 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/630278283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:08 smithi160 ceph-mon[34070]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:09.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T08:30:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:08 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/232394101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:30:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:08 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/630278283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:30:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:08 smithi188 ceph-mon[34645]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 -- 172.21.15.133:0/77517610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254109920 msgr2=0x7f02541101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 --2- 172.21.15.133:0/77517610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254109920 0x7f02541101b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f024c009be0 tx=0x7f024c02a6c0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 -- 172.21.15.133:0/77517610 shutdown_connections 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 --2- 172.21.15.133:0/77517610 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254109920 0x7f02541101b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 --2- 172.21.15.133:0/77517610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254108f60 0x7f02541093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 --2- 172.21.15.133:0/77517610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f0254108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 -- 172.21.15.133:0/77517610 >> 172.21.15.133:0/77517610 conn(0x7f0254076d00 msgr2=0x7f0254079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 -- 172.21.15.133:0/77517610 shutdown_connections 2024-05-09T08:30:09.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.368+0000 7f0258a00640 1 -- 172.21.15.133:0/77517610 wait complete. 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.369+0000 7f0258a00640 1 Processor -- start 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.369+0000 7f0258a00640 1 -- start start 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 0x7f025419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254109920 0x7f02541a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0254113980 con 0x7f0254109920 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0254113800 con 0x7f0254108f60 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0258a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0254113b00 con 0x7f0254107d70 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254109920 0x7f02541a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0253400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0252a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 0x7f025419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0253400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:59050/0 (socket says 172.21.15.133:59050) 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.370+0000 7f0253400640 1 -- 172.21.15.133:0/3987812163 learned_addr learned my addr 172.21.15.133:0/3987812163 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 -- 172.21.15.133:0/3987812163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 msgr2=0x7f025419e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 0x7f025419e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 -- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254109920 msgr2=0x7f02541a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254109920 0x7f02541a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 -- 172.21.15.133:0/3987812163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02541a5c40 con 0x7f0254107d70 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0252a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 0x7f025419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0253400640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f024400e9a0 tx=0x7f024400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f024be00640 1 -- 172.21.15.133:0/3987812163 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400cd80 con 0x7f0254107d70 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f024be00640 1 -- 172.21.15.133:0/3987812163 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244010910 con 0x7f0254107d70 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f024be00640 1 -- 172.21.15.133:0/3987812163 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244018a30 con 0x7f0254107d70 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02541a5f30 con 0x7f0254107d70 2024-05-09T08:30:09.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.371+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541a62e0 con 0x7f0254107d70 2024-05-09T08:30:09.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.375+0000 7f025cfd4740 1 Processor -- start 2024-05-09T08:30:09.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 -- start start 2024-05-09T08:30:09.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a6be03480 con 0x560a6bea5cf0 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560a6bdab5c0 con 0x560a6bea0190 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f025cfd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a6bdff440 con 0x560a6bea0b80 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f024be00640 1 -- 172.21.15.133:0/3987812163 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0244010ac0 con 0x7f0254107d70 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f0252a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f0252a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48458/0 (socket says 172.21.15.133:48458) 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f0252a00640 1 -- 172.21.15.133:0/3950619448 learned_addr learned my addr 172.21.15.133:0/3950619448 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.376+0000 7f0253e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0253400640 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 -- 172.21.15.133:0/3950619448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 msgr2=0x560a6bea5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 -- 172.21.15.133:0/3950619448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 msgr2=0x560a6bea05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 -- 172.21.15.133:0/3950619448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6bb172d0 con 0x560a6bea5cf0 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f0252a00640 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02400114f0 tx=0x7f02400119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27369eed491de307 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f023fe00640 1 -- 172.21.15.133:0/3950619448 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024001e070 con 0x560a6bea5cf0 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f023fe00640 1 -- 172.21.15.133:0/3950619448 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02400035f0 con 0x560a6bea5cf0 2024-05-09T08:30:09.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.377+0000 7f023fe00640 1 -- 172.21.15.133:0/3950619448 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024000f820 con 0x560a6bea5cf0 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f024be00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 0x7f02380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f024be00640 1 -- 172.21.15.133:0/3987812163 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f02440cf370 con 0x7f0254107d70 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 -- 172.21.15.133:0/3950619448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 msgr2=0x560a6bea8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02400114f0 tx=0x7f02400119b0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f0253e00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 0x7f02380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 -- 172.21.15.133:0/3950619448 shutdown_connections 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea8180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3950619448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 -- 172.21.15.133:0/3950619448 >> 172.21.15.133:0/3950619448 conn(0x560a6be678a0 msgr2=0x560a6be9bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 -- 172.21.15.133:0/3950619448 shutdown_connections 2024-05-09T08:30:09.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.378+0000 7f025cfd4740 1 -- 172.21.15.133:0/3950619448 wait complete. 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 -- start start 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f0253e00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 0x7f02380a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f024c005cc0 tx=0x7f024c005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 Processor -- start 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beaa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beb1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a6bea8ca0 con 0x560a6bea5cf0 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560a6bdbe980 con 0x560a6bea0b80 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a6bea8e20 con 0x560a6bea0190 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.379+0000 7f025cfd4740 10 client.0 _ll_register_callbacks cb 0x560a6bdf9fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beb1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beb1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48472/0 (socket says 172.21.15.133:48472) 2024-05-09T08:30:09.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:59074/0 (socket says 172.21.15.133:59074) 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253e00640 1 -- 172.21.15.133:0/3359406970 learned_addr learned my addr 172.21.15.133:0/3359406970 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0252a00640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beaa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 -- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 msgr2=0x560a6beaa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beaa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 -- 172.21.15.133:0/3359406970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 msgr2=0x560a6beb1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beb1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 -- 172.21.15.133:0/3359406970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6bbe16b0 con 0x560a6bea0190 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f0253400640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560a6bea15f0 tx=0x7f0244009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f0def743ba27c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f023cc00640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.380+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024409b6e0 con 0x560a6bea0190 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f0244018dd0 con 0x560a6bea0190 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02400035f0 con 0x560a6bea0190 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f024409b500 con 0x560a6bea0190 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 10 client.25165 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:09.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560a6beb4850 con 0x560a6bea0190 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02400035f0 con 0x560a6bea0190 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f024409aff0 con 0x560a6bea0190 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 10 client.25165 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 10 client.25165 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 20 client.25165 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6beb6e60 0x560a6beb82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 4 client.25165 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.381+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 --> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6be17060 con 0x560a6beb6e60 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.382+0000 7f0252a00640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6beb6e60 0x560a6beb82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.382+0000 7f0252a00640 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6beb6e60 0x560a6beb82d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f02ce24264c53738 server_cookie=610c13563b6444a6 in_seq=0 out_seq=0 2024-05-09T08:30:09.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.382+0000 7f023cc00640 10 client.25165 ms_handle_connect on v2:172.21.15.160:6838/145157679 2024-05-09T08:30:09.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f023cc00640 1 -- 172.21.15.133:0/3359406970 <== mds.0 v2:172.21.15.160:6838/145157679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a6be17060 con 0x560a6beb6e60 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f023cc00640 10 client.25165 handle_command_reply: tid=0 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 client.25165 shutdown 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 msgr2=0x560a6beaa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x560a6bea15f0 tx=0x7f0244009560 comp rx=0 tx=0).stop 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 shutdown_connections 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beb1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6beb6e60 0x560a6beb82d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beaa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3359406970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6beaa280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 >> 172.21.15.133:0/3359406970 conn(0x560a6be678a0 msgr2=0x560a6be9ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f023cc00640 0 client.25165 ms_handle_reset on v2:172.21.15.160:6838/145157679 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 shutdown_connections 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.383+0000 7f025cfd4740 1 -- 172.21.15.133:0/3359406970 wait complete. 2024-05-09T08:30:09.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.384+0000 7f025cfd4740 20 client.25165 trim_cache size 0 max 16384 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.402+0000 7f025cfd4740 1 Processor -- start 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 -- start start 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6be967d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beb1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb6610 con 0x560a6bea5cf0 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb6ce0 con 0x560a6bea0b80 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f025cfd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb73b0 con 0x560a6bea0190 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48488/0 (socket says 172.21.15.133:48488) 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f0253e00640 1 -- 172.21.15.133:0/3043968728 learned_addr learned my addr 172.21.15.133:0/3043968728 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f0253400640 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6be967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.403+0000 7f0252a00640 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beb1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 -- 172.21.15.133:0/3043968728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 msgr2=0x560a6be967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6be967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 -- 172.21.15.133:0/3043968728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 msgr2=0x560a6beb1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beb1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 -- 172.21.15.133:0/3043968728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6be9c8c0 con 0x560a6bea5cf0 2024-05-09T08:30:09.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f0253e00640 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f024c030f30 tx=0x7f024c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9409cdb1cf4752a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f023f400640 1 -- 172.21.15.133:0/3043968728 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c036050 con 0x560a6bea5cf0 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f023f400640 1 -- 172.21.15.133:0/3043968728 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024c037040 con 0x560a6bea5cf0 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.404+0000 7f023f400640 1 -- 172.21.15.133:0/3043968728 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c031820 con 0x560a6bea5cf0 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- 172.21.15.133:0/3043968728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 msgr2=0x560a6bea9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f024c030f30 tx=0x7f024c041000 comp rx=0 tx=0).stop 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- 172.21.15.133:0/3043968728 shutdown_connections 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6bea9e90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0b80 0x560a6beb1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 --2- 172.21.15.133:0/3043968728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0190 0x560a6be967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- 172.21.15.133:0/3043968728 >> 172.21.15.133:0/3043968728 conn(0x560a6be678a0 msgr2=0x560a6be9b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- 172.21.15.133:0/3043968728 shutdown_connections 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- 172.21.15.133:0/3043968728 wait complete. 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 -- start start 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.405+0000 7f025cfd4740 1 Processor -- start 2024-05-09T08:30:09.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb8150 con 0x560a6bea5cf0 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb7a80 con 0x560a6bea0190 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560a6beb8820 con 0x560a6bea0b80 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f025cfd4740 10 client.0 _ll_register_callbacks cb 0x7f024400ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48494/0 (socket says 172.21.15.133:48494) 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 -- 172.21.15.133:0/990674927 learned_addr learned my addr 172.21.15.133:0/990674927 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0252a00640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253400640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 -- 172.21.15.133:0/990674927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 msgr2=0x560a6bea4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 -- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 msgr2=0x560a6bea3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.406+0000 7f0253e00640 1 -- 172.21.15.133:0/990674927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6beb0340 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f0253400640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f0253e00640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f024c030d70 tx=0x7f024c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb820129ba4111d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f023cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c004240 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f024c0043e0 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024c037040 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f024c068070 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f025cfd4740 10 client.15078 fetch_fsmap learned FSMap version 22 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.407+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560a6beb1640 con 0x560a6bea5cf0 2024-05-09T08:30:09.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024c037040 con 0x560a6bea5cf0 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f024c03c040 con 0x560a6bea5cf0 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 10 client.15078 fetch_fsmap finished waiting for FSMap version 22 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 20 client.15078 populate_metadata read hostname 'smithi133' 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6bf20f20 0x560a6bf1fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 4 client.15078 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 --> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] -- command(tid 0: {"prefix": "status"}) -- 0x560a6be10bb0 con 0x560a6bf20f20 2024-05-09T08:30:09.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.408+0000 7f0253400640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6bf20f20 0x560a6bf1fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:09.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f0253400640 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6bf20f20 0x560a6bf1fc30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8685af6f4b6cd89d server_cookie=6dd3acd490a26bd3 in_seq=0 out_seq=0 2024-05-09T08:30:09.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f023cc00640 10 client.15078 ms_handle_connect on v2:172.21.15.160:6838/145157679 2024-05-09T08:30:09.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f023cc00640 1 -- 172.21.15.133:0/990674927 <== mds.0 v2:172.21.15.160:6838/145157679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560a6be10bb0 con 0x560a6bf20f20 2024-05-09T08:30:09.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f023cc00640 10 client.15078 handle_command_reply: tid=0 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 client.15078 shutdown 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 msgr2=0x560a6beafd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f024c030d70 tx=0x7f024c009480 comp rx=0 tx=0).stop 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 shutdown_connections 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a6bea5cf0 0x560a6beafd50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a6bea0b80 0x560a6bea4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:6838/145157679,v1:172.21.15.160:6839/145157679] conn(0x560a6bf20f20 0x560a6bf1fc30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 --2- 172.21.15.133:0/990674927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x560a6bea0190 0x560a6bea3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f023cc00640 0 client.15078 ms_handle_reset on v2:172.21.15.160:6838/145157679 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 >> 172.21.15.133:0/990674927 conn(0x560a6be678a0 msgr2=0x560a6bea9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 shutdown_connections 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.409+0000 7f025cfd4740 1 -- 172.21.15.133:0/990674927 wait complete. 2024-05-09T08:30:09.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f025cfd4740 20 client.15078 trim_cache size 0 max 16384 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "99f031e6-0ddc-11ef-bc98-c7b262605968", 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24566, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 133.192437716, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 22, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 86, 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 140.384958372 2024-05-09T08:30:09.415 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:30:09.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 msgr2=0x7f02380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 0x7f02380a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f024c005cc0 tx=0x7f024c005bd0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 msgr2=0x7f025419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:09.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f024400e9a0 tx=0x7f024400ee60 comp rx=0 tx=0).stop 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 shutdown_connections 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f02380a2c10 0x7f02380a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0254109920 0x7f02541a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0254108f60 0x7f025419e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 --2- 172.21.15.133:0/3987812163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0254107d70 0x7f025419e020 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 >> 172.21.15.133:0/3987812163 conn(0x7f0254076d00 msgr2=0x7f025410c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 shutdown_connections 2024-05-09T08:30:09.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:09.410+0000 7f0258a00640 1 -- 172.21.15.133:0/3987812163 wait complete. 2024-05-09T08:30:09.427 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T08:30:09.437 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T08:30:09.437 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T08:30:09.437 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T08:30:09.437 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T08:30:09.507 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T08:30:09.507 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T08:30:09.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:09.507 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-05-09T08:30:09.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:09.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-05-09T08:30:09.649 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-05-09T08:30:09.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:30:09.652 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-09T08:30:09.653 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T08:30:09.653 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:09.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-09T08:30:10.084 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:30:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:10 smithi133 ceph-mon[29610]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:11.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:10 smithi160 ceph-mon[34070]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:10 smithi188 ceph-mon[34645]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.312+0000 7f04b4600700 1 -- 172.21.15.133:0/3611380645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1083e0 msgr2=0x7f04ac1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.312+0000 7f04b4600700 1 --2- 172.21.15.133:0/3611380645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1083e0 0x7f04ac1087c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f04a00099c0 tx=0x7f04a0009cd0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.312+0000 7f04b4600700 1 -- 172.21.15.133:0/3611380645 shutdown_connections 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.312+0000 7f04b4600700 1 --2- 172.21.15.133:0/3611380645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.312+0000 7f04b4600700 1 --2- 172.21.15.133:0/3611380645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1083e0 0x7f04ac1087c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.313+0000 7f04b4600700 1 --2- 172.21.15.133:0/3611380645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1023d0 0x7f04ac102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.313+0000 7f04b4600700 1 -- 172.21.15.133:0/3611380645 >> 172.21.15.133:0/3611380645 conn(0x7f04ac079480 msgr2=0x7f04ac079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.313+0000 7f04b4600700 1 -- 172.21.15.133:0/3611380645 shutdown_connections 2024-05-09T08:30:12.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.313+0000 7f04b4600700 1 -- 172.21.15.133:0/3611380645 wait complete. 2024-05-09T08:30:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.314+0000 7f04b4600700 1 Processor -- start 2024-05-09T08:30:12.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.314+0000 7f04b4600700 1 -- start start 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 0x7f04ac1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1083e0 0x7f04ac19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac112690 con 0x7f04ac1023d0 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac112510 con 0x7f04ac102d90 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b4600700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac112810 con 0x7f04ac1083e0 2024-05-09T08:30:12.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 0x7f04ac1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 0x7f04ac1a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48508/0 (socket says 172.21.15.133:48508) 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1c00700 1 -- 172.21.15.133:0/1073853664 learned_addr learned my addr 172.21.15.133:0/1073853664 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1200700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b2600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1083e0 0x7f04ac19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1200700 1 -- 172.21.15.133:0/1073853664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1083e0 msgr2=0x7f04ac19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1200700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1083e0 0x7f04ac19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1200700 1 -- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 msgr2=0x7f04ac1a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.315+0000 7f04b1200700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 0x7f04ac1a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04b1200700 1 -- 172.21.15.133:0/1073853664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac19b1f0 con 0x7f04ac102d90 2024-05-09T08:30:12.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04b1200700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac1a0e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9f0 tx=0x7f049800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498004220 con 0x7f04ac102d90 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac19b4e0 con 0x7f04ac102d90 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498004d10 con 0x7f04ac102d90 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498005020 con 0x7f04ac102d90 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.316+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1a77a0 con 0x7f04ac102d90 2024-05-09T08:30:12.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.317+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c005330 con 0x7f04ac102d90 2024-05-09T08:30:12.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.319+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f049800c7c0 con 0x7f04ac102d90 2024-05-09T08:30:12.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.321+0000 7f04aa000700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 0x7f04940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.321+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f04980ceef0 con 0x7f04ac102d90 2024-05-09T08:30:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.321+0000 7f04b1c00700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 0x7f04940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:12.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.322+0000 7f04b1c00700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 0x7f04940a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:12.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.323+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f049809beb0 con 0x7f04ac102d90 2024-05-09T08:30:12.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.598+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f048c0058f0 con 0x7f04ac102d90 2024-05-09T08:30:12.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.599+0000 7f04aa000700 1 -- 172.21.15.133:0/1073853664 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7f049809b7b0 con 0x7f04ac102d90 2024-05-09T08:30:12.598 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T08:30:12.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.601+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 msgr2=0x7f04940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:12.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.601+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 0x7f04940a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009990 tx=0x7f04a0005960 comp rx=0 tx=0).stop 2024-05-09T08:30:12.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.601+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 msgr2=0x7f04ac1a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:12.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.601+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac1a0e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f049800e9f0 tx=0x7f049800ed00 comp rx=0 tx=0).stop 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.601+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 shutdown_connections 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f04940a2c20 0x7f04940a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04ac1083e0 0x7f04ac19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ac1023d0 0x7f04ac1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 --2- 172.21.15.133:0/1073853664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04ac102d90 0x7f04ac1a0e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 >> 172.21.15.133:0/1073853664 conn(0x7f04ac079480 msgr2=0x7f04ac074780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 shutdown_connections 2024-05-09T08:30:12.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:12.602+0000 7f04b4600700 1 -- 172.21.15.133:0/1073853664 wait complete. 2024-05-09T08:30:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:12 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1073853664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T08:30:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:12 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1073853664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T08:30:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:12 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1073853664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T08:30:13.417 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:30:13.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:13.418 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-05-09T08:30:13.475 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T08:30:13.475 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T08:30:13.476 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:30:13.477 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T08:30:13.477 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41972sec preferred_lft 41972sec 2024-05-09T08:30:13.477 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-05-09T08:30:13.477 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:30:13.477 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T08:30:13.477 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:30:13.477 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-05-09T08:30:13.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-05-09T08:30:13.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-05-09T08:30:13.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-05-09T08:30:13.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T08:30:13.478 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-05-09T08:30:13.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:13.879 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:13 smithi133 ceph-mon[29610]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:13.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:13 smithi160 ceph-mon[34070]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:13.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:13 smithi188 ceph-mon[34645]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:13.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:13.959 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T08:30:14.024 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-05-09T08:30:14.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:14.027 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-05-09T08:30:14.081 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-05-09T08:30:14.081 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T08:30:14.081 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T08:30:14.082 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T08:30:14.082 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T08:30:14.082 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-05-09T08:30:14.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:14.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:14.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:14.810 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-05-09T08:30:14.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:14.857 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-05-09T08:30:14.947 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:30:14.947 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-05-09T08:30:14.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:30:14.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T08:30:14.948 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-05-09T08:30:15.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:15.069 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:14 smithi133 ceph-mon[29610]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:15.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:15.121 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T08:30:15.121 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-05-09T08:30:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:14 smithi160 ceph-mon[34070]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:14 smithi188 ceph-mon[34645]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:15.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:15.410 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:30:15.410 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-05-09T08:30:15.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-05-09T08:30:15.411 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T08:30:15.411 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-05-09T08:30:15.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:15.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:15.604 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:30:15.604 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:15.647 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:30:15.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:30:15.647 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:15.736 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T08:30:15.736 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T08:30:15.737 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/9eeaffab-bdfb-4244-a54d-91ba89170d72 2024-05-09T08:30:15.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:15.737 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/9eeaffab-bdfb-4244-a54d-91ba89170d72 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-05-09T08:30:15.905 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-05-09T08:30:15.905 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-05-09T08:30:16.632 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-05-09T08:30:16.632 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-05-09T08:30:16.632 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.160:3300,172.21.15.188:3300" 2024-05-09T08:30:16.632 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@99f031e6-0ddc-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/9eeaffab-bdfb-4244-a54d-91ba89170d72 2024-05-09T08:30:16.632 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.133:3300/172.21.15.160:3300/172.21.15.188:3300" will pass to kernel 2024-05-09T08:30:16.637 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T08:30:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-05-09T08:30:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T08:30:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T08:30:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T08:30:16.641 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T08:30:16.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:30:16.642 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:16.729 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:30:16.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:30:16.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:16.858 INFO:teuthology.orchestra.run.smithi133.stdout:client15087 2024-05-09T08:30:16.858 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:30:16.858 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:30:16.907 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:30:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:16 smithi133 ceph-mon[29610]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:17.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:30:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:17.040 INFO:teuthology.orchestra.run.smithi133.stdout:client15087 2024-05-09T08:30:17.041 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:30:17.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:30:17.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:17.158 INFO:teuthology.orchestra.run.smithi133.stdout:client15087 2024-05-09T08:30:17.158 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968.client15087/status') 2024-05-09T08:30:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:16 smithi160 ceph-mon[34070]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:16 smithi188 ceph-mon[34645]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:30:17.289 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.15087 (3)192.168.0.1:0/2561899421 2024-05-09T08:30:17.289 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-05-09T08:30:17.289 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-05-09T08:30:17.289 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-05-09T08:30:17.289 INFO:teuthology.orchestra.run.smithi133.stderr:70 bytes copied, 7.7288e-05 s, 906 kB/s 2024-05-09T08:30:17.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:30:17.302 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T08:30:17.312 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T08:30:17.312 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T08:30:17.312 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 -- 172.21.15.133:0/2762331268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 msgr2=0x7f5260103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 --2- 172.21.15.133:0/2762331268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260103ed0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5248009900 tx=0x7f524802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 -- 172.21.15.133:0/2762331268 shutdown_connections 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 --2- 172.21.15.133:0/2762331268 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260105660 0x7f526010bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 --2- 172.21.15.133:0/2762331268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260104ca0 0x7f5260105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 --2- 172.21.15.133:0/2762331268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260103ed0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 -- 172.21.15.133:0/2762331268 >> 172.21.15.133:0/2762331268 conn(0x7f52600ff020 msgr2=0x7f5260101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 -- 172.21.15.133:0/2762331268 shutdown_connections 2024-05-09T08:30:17.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.510+0000 7f5267600640 1 -- 172.21.15.133:0/2762331268 wait complete. 2024-05-09T08:30:17.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.511+0000 7f5267600640 1 Processor -- start 2024-05-09T08:30:17.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.511+0000 7f5267600640 1 -- start start 2024-05-09T08:30:17.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 0x7f526019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 0x7f52601a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f526010f750 con 0x7f5260103ab0 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f526010f5d0 con 0x7f5260104ca0 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5267600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f526010f8d0 con 0x7f5260105660 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5264c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5257e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 0x7f526019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5265600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 0x7f52601a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5264c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50954/0 (socket says 172.21.15.133:50954) 2024-05-09T08:30:17.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.512+0000 7f5264c00640 1 -- 172.21.15.133:0/928732666 learned_addr learned my addr 172.21.15.133:0/928732666 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 -- 172.21.15.133:0/928732666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 msgr2=0x7f52601a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 0x7f52601a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 -- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 msgr2=0x7f5260199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 -- 172.21.15.133:0/928732666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52601a19b0 con 0x7f5260104ca0 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5265600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 0x7f52601a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5257e00640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 0x7f526019a2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f525000d6d0 tx=0x7f525000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250017070 con 0x7f5260104ca0 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52601a1ca0 con 0x7f5260104ca0 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5250009820 con 0x7f5260104ca0 2024-05-09T08:30:17.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250004cd0 con 0x7f5260104ca0 2024-05-09T08:30:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.513+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52601a21a0 con 0x7f5260104ca0 2024-05-09T08:30:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.515+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526010ad00 con 0x7f5260104ca0 2024-05-09T08:30:17.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.517+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f52500099d0 con 0x7f5260104ca0 2024-05-09T08:30:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.518+0000 7f525e000640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 0x7f52400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:17.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.518+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f52500cebf0 con 0x7f5260104ca0 2024-05-09T08:30:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.518+0000 7f5264c00640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 0x7f52400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:17.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.519+0000 7f5264c00640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 0x7f52400a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:17.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.522+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f525009b4d0 con 0x7f5260104ca0 2024-05-09T08:30:17.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.767+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f52600624d0 con 0x7f5260104ca0 2024-05-09T08:30:17.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.822+0000 7f525e000640 1 -- 172.21.15.133:0/928732666 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f525009f7d0 con 0x7f5260104ca0 2024-05-09T08:30:17.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.825+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 msgr2=0x7f52400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:17.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.825+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 0x7f52400a50f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5248006010 tx=0x7f52480315b0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.825+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 msgr2=0x7f526019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:17.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.825+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 0x7f526019a2e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f525000d6d0 tx=0x7f525000db90 comp rx=0 tx=0).stop 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 shutdown_connections 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f52400a2c30 0x7f52400a50f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5260105660 0x7f52601a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5260104ca0 0x7f526019a2e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 --2- 172.21.15.133:0/928732666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5260103ab0 0x7f5260199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 >> 172.21.15.133:0/928732666 conn(0x7f52600ff020 msgr2=0x7f5260107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 shutdown_connections 2024-05-09T08:30:17.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:17.826+0000 7f5267600640 1 -- 172.21.15.133:0/928732666 wait complete. 2024-05-09T08:30:17.848 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-09T08:30:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 -- 172.21.15.133:0/1511621730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 msgr2=0x7f1b4c108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 --2- 172.21.15.133:0/1511621730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c108100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c02a3f0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 -- 172.21.15.133:0/1511621730 shutdown_connections 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 --2- 172.21.15.133:0/1511621730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c109890 0x7f1b4c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 --2- 172.21.15.133:0/1511621730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 --2- 172.21.15.133:0/1511621730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c108100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 -- 172.21.15.133:0/1511621730 >> 172.21.15.133:0/1511621730 conn(0x7f1b4c076cc0 msgr2=0x7f1b4c0790e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.043+0000 7f1b4b200640 1 -- 172.21.15.133:0/1511621730 shutdown_connections 2024-05-09T08:30:18.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.044+0000 7f1b4b200640 1 -- 172.21.15.133:0/1511621730 wait complete. 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.044+0000 7f1b4b200640 1 Processor -- start 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 -- start start 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c109890 0x7f1b4c1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1139a0 con 0x7f1b4c107ce0 2024-05-09T08:30:18.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c113820 con 0x7f1b4c108ed0 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c113b20 con 0x7f1b4c109890 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b49e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b49e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50964/0 (socket says 172.21.15.133:50964) 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b49e00640 1 -- 172.21.15.133:0/4294073293 learned_addr learned my addr 172.21.15.133:0/4294073293 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b41400640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.045+0000 7f1b4a800640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c109890 0x7f1b4c1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 -- 172.21.15.133:0/4294073293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c109890 msgr2=0x7f1b4c1a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c109890 0x7f1b4c1a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 -- 172.21.15.133:0/4294073293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 msgr2=0x7f1b4c19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 -- 172.21.15.133:0/4294073293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c1a5bd0 con 0x7f1b4c107ce0 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b41400640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:30:18.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b49e00640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c031440 tx=0x7f1b3c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c03f030 con 0x7f1b4c107ce0 2024-05-09T08:30:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c003ce0 con 0x7f1b4c107ce0 2024-05-09T08:30:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c035070 con 0x7f1b4c107ce0 2024-05-09T08:30:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.046+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1a5e60 con 0x7f1b4c107ce0 2024-05-09T08:30:18.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.047+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1a6270 con 0x7f1b4c107ce0 2024-05-09T08:30:18.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.048+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c10efa0 con 0x7f1b4c107ce0 2024-05-09T08:30:18.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.050+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1b3c042030 con 0x7f1b4c107ce0 2024-05-09T08:30:18.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.051+0000 7f1b43400640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 0x7f1b300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:30:18.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.051+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1b3c0ef500 con 0x7f1b4c107ce0 2024-05-09T08:30:18.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.051+0000 7f1b41400640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 0x7f1b300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:30:18.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.052+0000 7f1b41400640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 0x7f1b300a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c19f4c0 tx=0x7f1b34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:30:18.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.056+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b3c0bbde0 con 0x7f1b4c107ce0 2024-05-09T08:30:18.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:17 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/928732666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:17 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:17 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/928732666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:17 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:17 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/928732666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:17 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-09T08:30:18.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.289+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 --> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f1b4c061dd0 con 0x7f1b300a2b80 2024-05-09T08:30:18.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.317+0000 7f1b43400640 1 -- 172.21.15.133:0/4294073293 <== mgr.14152 v2:172.21.15.133:6800/2058981411 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f1b4c061dd0 con 0x7f1b300a2b80 2024-05-09T08:30:18.317 INFO:teuthology.orchestra.run.smithi133.stdout:{} 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 msgr2=0x7f1b300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 0x7f1b300a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c19f4c0 tx=0x7f1b34005f90 comp rx=0 tx=0).stop 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 msgr2=0x7f1b4c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c031440 tx=0x7f1b3c032ce0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 shutdown_connections 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f1b300a2b80 0x7f1b300a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b4c109890 0x7f1b4c1a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c108ed0 0x7f1b4c19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 --2- 172.21.15.133:0/4294073293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b4c107ce0 0x7f1b4c19dfb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:30:18.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.320+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 >> 172.21.15.133:0/4294073293 conn(0x7f1b4c076cc0 msgr2=0x7f1b4c10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:30:18.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.321+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 shutdown_connections 2024-05-09T08:30:18.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:30:18.321+0000 7f1b4b200640 1 -- 172.21.15.133:0/4294073293 wait complete. 2024-05-09T08:30:18.339 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T08:30:18.352 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T08:30:18.369 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:30:18.369 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T08:30:18.369 INFO:tasks.workunit:timeout=3h 2024-05-09T08:30:18.370 INFO:tasks.workunit:cleanup=True 2024-05-09T08:30:18.370 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:18.419 INFO:teuthology.orchestra.run.smithi133.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:18.419 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T08:30:18.419 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-09T08:30:18.419 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:30:18.420 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T08:30:18.420 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-05-09 08:28:03.219480391 +0000 2024-05-09T08:30:18.420 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-05-09 08:28:03.219480391 +0000 2024-05-09T08:30:18.420 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-05-09 08:30:16.718791383 +0000 2024-05-09T08:30:18.420 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-05-09 08:28:03.219480391 +0000 2024-05-09T08:30:18.420 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T08:30:18.420 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T08:30:18.517 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:30:18.576 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T08:30:18.678 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:30:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:30:18.300+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:30:18.678 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:30:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:30:18.300+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:30:18.678 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:30:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:30:18.300+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:30:18.678 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:30:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:30:18.300+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:30:18.678 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:30:18 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:30:18.301+0000 7ff20d400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T08:30:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:18 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T08:30:19.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:18 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:30:19.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:18 smithi160 ceph-mon[34070]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:30:19.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:18 smithi160 ceph-mon[34070]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:18 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T08:30:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:18 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:30:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:18 smithi133 ceph-mon[29610]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:30:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:18 smithi133 ceph-mon[29610]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:18 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-09T08:30:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:18 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:30:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:18 smithi188 ceph-mon[34645]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T08:30:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:18 smithi188 ceph-mon[34645]: pgmap v389: 97 pgs: 97 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:20 smithi160 ceph-mon[34070]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T08:30:21.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:20 smithi160 ceph-mon[34070]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:30:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:20 smithi133 ceph-mon[29610]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T08:30:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:20 smithi133 ceph-mon[29610]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:30:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:20 smithi188 ceph-mon[34645]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T08:30:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:20 smithi188 ceph-mon[34645]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:30:22.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:30:22.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:30:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:30:23.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:23 smithi133 ceph-mon[29610]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:23.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:23.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:23 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:23.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:23 smithi188 ceph-mon[34645]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:23.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:23.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:23 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:24.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:23 smithi160 ceph-mon[34070]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:24.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:24.017 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:23 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:25 smithi160 ceph-mon[34070]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:25.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:25 smithi133 ceph-mon[29610]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:25.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:25.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:25 smithi188 ceph-mon[34645]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-09T08:30:27.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:27 smithi160 ceph-mon[34070]: pgmap v393: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:27.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:27 smithi133 ceph-mon[29610]: pgmap v393: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:27.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:27.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:27 smithi188 ceph-mon[34645]: pgmap v393: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:28.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:30:28.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:30:28.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:28.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:30:28.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:30:28.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:28.686 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:30:28.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:30:28.687 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:30:29.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:29 smithi160 ceph-mon[34070]: pgmap v394: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T08:30:29.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:29 smithi133 ceph-mon[29610]: pgmap v394: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T08:30:29.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:29 smithi188 ceph-mon[34645]: pgmap v394: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-09T08:30:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:30 smithi160 ceph-mon[34070]: pgmap v395: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:30 smithi133 ceph-mon[29610]: pgmap v395: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:30 smithi188 ceph-mon[34645]: pgmap v395: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-09T08:30:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:32 smithi160 ceph-mon[34070]: pgmap v396: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-09T08:30:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:32 smithi133 ceph-mon[29610]: pgmap v396: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-09T08:30:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:32 smithi188 ceph-mon[34645]: pgmap v396: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-05-09T08:30:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:34 smithi160 ceph-mon[34070]: pgmap v397: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:34 smithi133 ceph-mon[29610]: pgmap v397: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:34 smithi188 ceph-mon[34645]: pgmap v397: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:36 smithi160 ceph-mon[34070]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:36 smithi133 ceph-mon[29610]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:36 smithi188 ceph-mon[34645]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-09T08:30:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:38 smithi160 ceph-mon[34070]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:38 smithi133 ceph-mon[29610]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:38 smithi188 ceph-mon[34645]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:40 smithi160 ceph-mon[34070]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:41.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:40 smithi133 ceph-mon[29610]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:40 smithi188 ceph-mon[34645]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:42 smithi160 ceph-mon[34070]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:42 smithi133 ceph-mon[29610]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:42 smithi188 ceph-mon[34645]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:44 smithi160 ceph-mon[34070]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:44 smithi133 ceph-mon[29610]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:44 smithi188 ceph-mon[34645]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:46 smithi160 ceph-mon[34070]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:46 smithi133 ceph-mon[29610]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:46 smithi188 ceph-mon[34645]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:30:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:48 smithi160 ceph-mon[34070]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:48 smithi133 ceph-mon[29610]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:48 smithi188 ceph-mon[34645]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:30:51.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:50 smithi160 ceph-mon[34070]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:50 smithi133 ceph-mon[29610]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:50 smithi188 ceph-mon[34645]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:30:53.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:53 smithi160 ceph-mon[34070]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:30:53.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:53 smithi133 ceph-mon[29610]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:30:53.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:53 smithi188 ceph-mon[34645]: pgmap v406: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:30:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:54 smithi160 ceph-mon[34070]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:54 smithi133 ceph-mon[29610]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:54 smithi188 ceph-mon[34645]: pgmap v407: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:56 smithi160 ceph-mon[34070]: pgmap v408: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:57.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:56 smithi133 ceph-mon[29610]: pgmap v408: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:56 smithi188 ceph-mon[34645]: pgmap v408: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:30:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:30:58 smithi160 ceph-mon[34070]: pgmap v409: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T08:30:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:30:58 smithi133 ceph-mon[29610]: pgmap v409: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T08:30:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:30:58 smithi188 ceph-mon[34645]: pgmap v409: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-09T08:31:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:00 smithi160 ceph-mon[34070]: pgmap v410: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:00 smithi133 ceph-mon[29610]: pgmap v410: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:00 smithi188 ceph-mon[34645]: pgmap v410: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:02 smithi160 ceph-mon[34070]: pgmap v411: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:02 smithi133 ceph-mon[29610]: pgmap v411: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:02 smithi188 ceph-mon[34645]: pgmap v411: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T08:31:05.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:04 smithi160 ceph-mon[34070]: pgmap v412: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T08:31:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:04 smithi133 ceph-mon[29610]: pgmap v412: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T08:31:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:04 smithi188 ceph-mon[34645]: pgmap v412: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T08:31:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:06 smithi160 ceph-mon[34070]: pgmap v413: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:06 smithi133 ceph-mon[29610]: pgmap v413: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:06 smithi188 ceph-mon[34645]: pgmap v413: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:08 smithi160 ceph-mon[34070]: pgmap v414: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:08 smithi133 ceph-mon[29610]: pgmap v414: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:08 smithi188 ceph-mon[34645]: pgmap v414: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:10 smithi160 ceph-mon[34070]: pgmap v415: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:10 smithi133 ceph-mon[29610]: pgmap v415: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:10 smithi188 ceph-mon[34645]: pgmap v415: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:12 smithi160 ceph-mon[34070]: pgmap v416: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:12 smithi133 ceph-mon[29610]: pgmap v416: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:12 smithi188 ceph-mon[34645]: pgmap v416: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:14 smithi160 ceph-mon[34070]: pgmap v417: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:14 smithi133 ceph-mon[29610]: pgmap v417: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:14 smithi188 ceph-mon[34645]: pgmap v417: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:16 smithi160 ceph-mon[34070]: pgmap v418: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:17.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:16 smithi133 ceph-mon[29610]: pgmap v418: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:16 smithi188 ceph-mon[34645]: pgmap v418: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:18 smithi160 ceph-mon[34070]: pgmap v419: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:18 smithi133 ceph-mon[29610]: pgmap v419: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:18 smithi188 ceph-mon[34645]: pgmap v419: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:20 smithi160 ceph-mon[34070]: pgmap v420: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:20 smithi133 ceph-mon[29610]: pgmap v420: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:20 smithi188 ceph-mon[34645]: pgmap v420: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:22 smithi160 ceph-mon[34070]: pgmap v421: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:23.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:22 smithi133 ceph-mon[29610]: pgmap v421: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:22 smithi188 ceph-mon[34645]: pgmap v421: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:24 smithi160 ceph-mon[34070]: pgmap v422: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:25.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:24 smithi133 ceph-mon[29610]: pgmap v422: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:24 smithi188 ceph-mon[34645]: pgmap v422: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:26 smithi160 ceph-mon[34070]: pgmap v423: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:26 smithi133 ceph-mon[29610]: pgmap v423: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:26 smithi188 ceph-mon[34645]: pgmap v423: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-09T08:31:28.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:27 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:31:28.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:27 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:31:28.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:27 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:31:29.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-09T08:31:29.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.164 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.165 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:28 smithi160 ceph-mon[34070]: pgmap v424: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-09T08:31:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:28 smithi133 ceph-mon[29610]: pgmap v424: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:28 smithi188 ceph-mon[34645]: pgmap v424: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T08:31:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:31:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:31:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:30.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:31:30.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:31:30.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:31:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:31:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:31:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:30 smithi160 ceph-mon[34070]: pgmap v425: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:30 smithi133 ceph-mon[29610]: pgmap v425: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:30 smithi188 ceph-mon[34645]: pgmap v425: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:32 smithi160 ceph-mon[34070]: pgmap v426: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:32 smithi133 ceph-mon[29610]: pgmap v426: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:32 smithi188 ceph-mon[34645]: pgmap v426: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:34 smithi160 ceph-mon[34070]: pgmap v427: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:34 smithi133 ceph-mon[29610]: pgmap v427: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:34 smithi188 ceph-mon[34645]: pgmap v427: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:36 smithi160 ceph-mon[34070]: pgmap v428: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:37.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:36 smithi133 ceph-mon[29610]: pgmap v428: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:36 smithi188 ceph-mon[34645]: pgmap v428: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:38 smithi160 ceph-mon[34070]: pgmap v429: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T08:31:39.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:38 smithi133 ceph-mon[29610]: pgmap v429: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T08:31:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:38 smithi188 ceph-mon[34645]: pgmap v429: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-09T08:31:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:40 smithi160 ceph-mon[34070]: pgmap v430: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:41.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:40 smithi133 ceph-mon[29610]: pgmap v430: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:40 smithi188 ceph-mon[34645]: pgmap v430: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:31:43.030 INFO:tasks.workunit.client.0.smithi133.stderr:Updating files: 63% (7930/12535) Updating files: 64% (8023/12535) Updating files: 65% (8148/12535) Updating files: 66% (8274/12535) Updating files: 67% (8399/12535) Updating files: 68% (8524/12535) Updating files: 69% (8650/12535) Updating files: 70% (8775/12535) Updating files: 71% (8900/12535) Updating files: 72% (9026/12535) Updating files: 73% (9151/12535) Updating files: 74% (9276/12535) Updating files: 75% (9402/12535) Updating files: 76% (9527/12535) Updating files: 77% (9652/12535) Updating files: 78% (9778/12535) Updating files: 79% (9903/12535) Updating files: 80% (10028/12535) Updating files: 81% (10154/12535) Updating files: 82% (10279/12535) Updating files: 83% (10405/12535) Updating files: 84% (10530/12535) Updating files: 85% (10655/12535) Updating files: 86% (10781/12535) Updating files: 87% (10906/12535) Updating files: 88% (11031/12535) Updating files: 89% (11157/12535) Updating files: 90% (11282/12535) Updating files: 91% (11407/12535) Updating files: 92% (11533/12535) Updating files: 93% (11658/12535) Updating files: 94% (11783/12535) Updating files: 95% (11909/12535) Updating files: 96% (12034/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T08:31:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:42 smithi160 ceph-mon[34070]: pgmap v431: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:42 smithi133 ceph-mon[29610]: pgmap v431: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:42 smithi188 ceph-mon[34645]: pgmap v431: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:31:43.432 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T08:31:43.432 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T08:31:43.433 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.434 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T08:31:43.435 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-05-09T08:31:43.435 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T08:31:43.442 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T08:31:43.505 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T08:31:43.512 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:31:43.512 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T08:31:44.375 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T08:31:44.481 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T08:31:44.562 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:31:44.566 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:31:44.566 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T08:31:44.645 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:31:44.654 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:31:44.654 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T08:31:44.704 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-09T08:31:44.705 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-09T08:31:44.706 DEBUG:teuthology.orchestra.run.smithi133:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-09T08:31:44.778 INFO:tasks.workunit.client.0.smithi133.stderr:+ dbench 1 2024-05-09T08:31:44.798 INFO:tasks.workunit.client.0.smithi133.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-09T08:31:44.798 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:31:44.799 INFO:tasks.workunit.client.0.smithi133.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-09T08:31:44.802 INFO:tasks.workunit.client.0.smithi133.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-09T08:31:44.903 INFO:tasks.workunit.client.0.smithi133.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-09T08:31:44.903 INFO:tasks.workunit.client.0.smithi133.stdout:releasing clients 2024-05-09T08:31:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:44 smithi160 ceph-mon[34070]: pgmap v432: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:44 smithi133 ceph-mon[29610]: pgmap v432: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:44 smithi188 ceph-mon[34645]: pgmap v432: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-09T08:31:45.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 2197 26.75 MB/sec warmup 1 sec latency 12.446 ms 2024-05-09T08:31:46.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 4236 20.13 MB/sec warmup 2 sec latency 11.263 ms 2024-05-09T08:31:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:46 smithi160 ceph-mon[34070]: pgmap v433: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 105 KiB/s wr, 24 op/s 2024-05-09T08:31:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:46 smithi133 ceph-mon[29610]: pgmap v433: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 105 KiB/s wr, 24 op/s 2024-05-09T08:31:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:46 smithi188 ceph-mon[34645]: pgmap v433: 97 pgs: 97 active+clean; 1.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 105 KiB/s wr, 24 op/s 2024-05-09T08:31:47.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 6927 18.19 MB/sec warmup 3 sec latency 10.264 ms 2024-05-09T08:31:48.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 8797 16.11 MB/sec warmup 4 sec latency 11.046 ms 2024-05-09T08:31:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:48 smithi160 ceph-mon[34070]: pgmap v434: 97 pgs: 97 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 158 KiB/s wr, 26 op/s 2024-05-09T08:31:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:48 smithi133 ceph-mon[29610]: pgmap v434: 97 pgs: 97 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 158 KiB/s wr, 26 op/s 2024-05-09T08:31:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:48 smithi188 ceph-mon[34645]: pgmap v434: 97 pgs: 97 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 158 KiB/s wr, 26 op/s 2024-05-09T08:31:49.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 11081 15.66 MB/sec warmup 5 sec latency 12.861 ms 2024-05-09T08:31:50.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13048 14.22 MB/sec warmup 6 sec latency 24.056 ms 2024-05-09T08:31:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:50 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:31:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:50 smithi160 ceph-mon[34070]: pgmap v435: 97 pgs: 97 active+clean; 3.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 274 KiB/s wr, 31 op/s 2024-05-09T08:31:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:50 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:31:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:50 smithi133 ceph-mon[29610]: pgmap v435: 97 pgs: 97 active+clean; 3.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 274 KiB/s wr, 31 op/s 2024-05-09T08:31:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:50 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:31:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:50 smithi188 ceph-mon[34645]: pgmap v435: 97 pgs: 97 active+clean; 3.5 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 274 KiB/s wr, 31 op/s 2024-05-09T08:31:51.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 13054 12.19 MB/sec warmup 7 sec latency 984.877 ms 2024-05-09T08:31:52.904 INFO:tasks.workunit.client.0.smithi133.stdout: 1 14449 12.14 MB/sec warmup 8 sec latency 1151.446 ms 2024-05-09T08:31:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:52 smithi160 ceph-mon[34070]: pgmap v436: 97 pgs: 97 active+clean; 6.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 592 KiB/s wr, 77 op/s 2024-05-09T08:31:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:52 smithi133 ceph-mon[29610]: pgmap v436: 97 pgs: 97 active+clean; 6.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 592 KiB/s wr, 77 op/s 2024-05-09T08:31:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:52 smithi188 ceph-mon[34645]: pgmap v436: 97 pgs: 97 active+clean; 6.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 592 KiB/s wr, 77 op/s 2024-05-09T08:31:53.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16174 11.44 MB/sec warmup 9 sec latency 11.596 ms 2024-05-09T08:31:54.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16596 10.44 MB/sec warmup 10 sec latency 743.000 ms 2024-05-09T08:31:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:54 smithi160 ceph-mon[34070]: pgmap v437: 97 pgs: 97 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-05-09T08:31:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:54 smithi133 ceph-mon[29610]: pgmap v437: 97 pgs: 97 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-05-09T08:31:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:54 smithi188 ceph-mon[34645]: pgmap v437: 97 pgs: 97 active+clean; 7.4 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 718 KiB/s wr, 93 op/s 2024-05-09T08:31:55.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16596 9.49 MB/sec warmup 11 sec latency 1743.180 ms 2024-05-09T08:31:56.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16596 8.70 MB/sec warmup 12 sec latency 2743.272 ms 2024-05-09T08:31:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:56 smithi160 ceph-mon[34070]: pgmap v438: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 809 KiB/s wr, 105 op/s 2024-05-09T08:31:57.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:56 smithi133 ceph-mon[29610]: pgmap v438: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 809 KiB/s wr, 105 op/s 2024-05-09T08:31:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:56 smithi188 ceph-mon[34645]: pgmap v438: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 366 KiB/s rd, 809 KiB/s wr, 105 op/s 2024-05-09T08:31:57.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 8.04 MB/sec warmup 13 sec latency 2908.524 ms 2024-05-09T08:31:58.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 7.47 MB/sec warmup 14 sec latency 1791.989 ms 2024-05-09T08:31:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:31:58 smithi160 ceph-mon[34070]: pgmap v439: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 705 KiB/s wr, 86 op/s 2024-05-09T08:31:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:31:58 smithi133 ceph-mon[29610]: pgmap v439: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 705 KiB/s wr, 86 op/s 2024-05-09T08:31:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:31:58 smithi188 ceph-mon[34645]: pgmap v439: 97 pgs: 97 active+clean; 8.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 705 KiB/s wr, 86 op/s 2024-05-09T08:31:59.905 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 6.97 MB/sec warmup 15 sec latency 2792.136 ms 2024-05-09T08:32:00.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 6.54 MB/sec warmup 16 sec latency 3792.307 ms 2024-05-09T08:32:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:00 smithi160 ceph-mon[34070]: pgmap v440: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 666 KiB/s wr, 87 op/s 2024-05-09T08:32:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:00 smithi133 ceph-mon[29610]: pgmap v440: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 666 KiB/s wr, 87 op/s 2024-05-09T08:32:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:00 smithi188 ceph-mon[34645]: pgmap v440: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 666 KiB/s wr, 87 op/s 2024-05-09T08:32:01.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16624 6.15 MB/sec warmup 17 sec latency 4792.475 ms 2024-05-09T08:32:02.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 5.84 MB/sec warmup 18 sec latency 4964.436 ms 2024-05-09T08:32:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:02 smithi160 ceph-mon[34070]: pgmap v441: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 550 KiB/s wr, 84 op/s 2024-05-09T08:32:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:02 smithi133 ceph-mon[29610]: pgmap v441: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 550 KiB/s wr, 84 op/s 2024-05-09T08:32:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:02 smithi188 ceph-mon[34645]: pgmap v441: 97 pgs: 97 active+clean; 8.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 550 KiB/s wr, 84 op/s 2024-05-09T08:32:03.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 5.54 MB/sec warmup 19 sec latency 1699.981 ms 2024-05-09T08:32:04.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 5.26 MB/sec warmup 20 sec latency 2700.149 ms 2024-05-09T08:32:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:04 smithi160 ceph-mon[34070]: pgmap v442: 97 pgs: 97 active+clean; 8.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 248 KiB/s wr, 40 op/s 2024-05-09T08:32:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:04 smithi133 ceph-mon[29610]: pgmap v442: 97 pgs: 97 active+clean; 8.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 248 KiB/s wr, 40 op/s 2024-05-09T08:32:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:04 smithi188 ceph-mon[34645]: pgmap v442: 97 pgs: 97 active+clean; 8.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 248 KiB/s wr, 40 op/s 2024-05-09T08:32:05.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 5.01 MB/sec warmup 21 sec latency 3700.298 ms 2024-05-09T08:32:06.906 INFO:tasks.workunit.client.0.smithi133.stdout: 1 16685 4.78 MB/sec warmup 22 sec latency 4700.457 ms 2024-05-09T08:32:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:06 smithi160 ceph-mon[34070]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 656 KiB/s wr, 26 op/s 2024-05-09T08:32:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:06 smithi133 ceph-mon[29610]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 656 KiB/s wr, 26 op/s 2024-05-09T08:32:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:06 smithi188 ceph-mon[34645]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 656 KiB/s wr, 26 op/s 2024-05-09T08:32:07.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 18149 5.07 MB/sec warmup 23 sec latency 4887.057 ms 2024-05-09T08:32:08.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20130 5.15 MB/sec warmup 24 sec latency 26.698 ms 2024-05-09T08:32:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:08 smithi160 ceph-mon[34070]: pgmap v444: 97 pgs: 97 active+clean; 22 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-09T08:32:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:08 smithi133 ceph-mon[29610]: pgmap v444: 97 pgs: 97 active+clean; 22 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-09T08:32:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:08 smithi188 ceph-mon[34645]: pgmap v444: 97 pgs: 97 active+clean; 22 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.2 MiB/s wr, 16 op/s 2024-05-09T08:32:09.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20173 4.94 MB/sec warmup 25 sec latency 937.242 ms 2024-05-09T08:32:10.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20173 4.75 MB/sec warmup 26 sec latency 1937.385 ms 2024-05-09T08:32:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:10 smithi160 ceph-mon[34070]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 43 op/s 2024-05-09T08:32:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:10 smithi133 ceph-mon[29610]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 43 op/s 2024-05-09T08:32:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:10 smithi188 ceph-mon[34645]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 43 op/s 2024-05-09T08:32:11.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20173 4.58 MB/sec warmup 27 sec latency 2937.488 ms 2024-05-09T08:32:12.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20213 4.42 MB/sec warmup 28 sec latency 3144.407 ms 2024-05-09T08:32:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:12 smithi160 ceph-mon[34070]: pgmap v446: 97 pgs: 97 active+clean; 36 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 52 op/s 2024-05-09T08:32:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:12 smithi133 ceph-mon[29610]: pgmap v446: 97 pgs: 97 active+clean; 36 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 52 op/s 2024-05-09T08:32:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:12 smithi188 ceph-mon[34645]: pgmap v446: 97 pgs: 97 active+clean; 36 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 52 op/s 2024-05-09T08:32:13.907 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20213 4.27 MB/sec warmup 29 sec latency 1712.437 ms 2024-05-09T08:32:14.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20213 4.12 MB/sec warmup 30 sec latency 2712.622 ms 2024-05-09T08:32:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:14 smithi160 ceph-mon[34070]: pgmap v447: 97 pgs: 97 active+clean; 35 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.9 MiB/s wr, 75 op/s 2024-05-09T08:32:15.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:14 smithi133 ceph-mon[29610]: pgmap v447: 97 pgs: 97 active+clean; 35 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.9 MiB/s wr, 75 op/s 2024-05-09T08:32:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:14 smithi188 ceph-mon[34645]: pgmap v447: 97 pgs: 97 active+clean; 35 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 2.9 MiB/s wr, 75 op/s 2024-05-09T08:32:15.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20213 3.99 MB/sec warmup 31 sec latency 3712.787 ms 2024-05-09T08:32:16.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 20213 3.87 MB/sec warmup 32 sec latency 4712.953 ms 2024-05-09T08:32:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:16 smithi160 ceph-mon[34070]: pgmap v448: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-05-09T08:32:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:16 smithi133 ceph-mon[29610]: pgmap v448: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-05-09T08:32:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:16 smithi188 ceph-mon[34645]: pgmap v448: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-05-09T08:32:17.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 21617 4.10 MB/sec warmup 33 sec latency 4885.746 ms 2024-05-09T08:32:18.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23525 4.16 MB/sec warmup 34 sec latency 20.404 ms 2024-05-09T08:32:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:18 smithi160 ceph-mon[34070]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 2.3 MiB/s wr, 79 op/s 2024-05-09T08:32:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:18 smithi133 ceph-mon[29610]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 2.3 MiB/s wr, 79 op/s 2024-05-09T08:32:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:18 smithi188 ceph-mon[34645]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 2.3 MiB/s wr, 79 op/s 2024-05-09T08:32:19.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23660 4.07 MB/sec warmup 35 sec latency 926.111 ms 2024-05-09T08:32:20.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23660 3.96 MB/sec warmup 36 sec latency 1926.255 ms 2024-05-09T08:32:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:20 smithi160 ceph-mon[34070]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-09T08:32:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:20 smithi133 ceph-mon[29610]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-09T08:32:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:20 smithi188 ceph-mon[34645]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.8 MiB/s wr, 97 op/s 2024-05-09T08:32:21.908 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23660 3.85 MB/sec warmup 37 sec latency 2926.398 ms 2024-05-09T08:32:22.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23754 3.76 MB/sec warmup 38 sec latency 3089.000 ms 2024-05-09T08:32:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:22 smithi160 ceph-mon[34070]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-05-09T08:32:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:22 smithi133 ceph-mon[29610]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-05-09T08:32:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:22 smithi188 ceph-mon[34645]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.2 MiB/s wr, 73 op/s 2024-05-09T08:32:23.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23754 3.66 MB/sec warmup 39 sec latency 1662.371 ms 2024-05-09T08:32:24.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23754 3.57 MB/sec warmup 40 sec latency 2662.523 ms 2024-05-09T08:32:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:24 smithi160 ceph-mon[34070]: pgmap v452: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 636 KiB/s wr, 65 op/s 2024-05-09T08:32:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:24 smithi133 ceph-mon[29610]: pgmap v452: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 636 KiB/s wr, 65 op/s 2024-05-09T08:32:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:24 smithi188 ceph-mon[34645]: pgmap v452: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 636 KiB/s wr, 65 op/s 2024-05-09T08:32:25.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23754 3.48 MB/sec warmup 41 sec latency 3662.686 ms 2024-05-09T08:32:26.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23754 3.40 MB/sec warmup 42 sec latency 4662.863 ms 2024-05-09T08:32:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:26 smithi160 ceph-mon[34070]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 179 KiB/s wr, 41 op/s 2024-05-09T08:32:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:26 smithi133 ceph-mon[29610]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 179 KiB/s wr, 41 op/s 2024-05-09T08:32:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:26 smithi188 ceph-mon[34645]: pgmap v453: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 179 KiB/s wr, 41 op/s 2024-05-09T08:32:27.909 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23815 3.34 MB/sec warmup 43 sec latency 4839.558 ms 2024-05-09T08:32:28.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23815 3.26 MB/sec warmup 44 sec latency 1684.474 ms 2024-05-09T08:32:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:28 smithi160 ceph-mon[34070]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 186 KiB/s wr, 33 op/s 2024-05-09T08:32:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:28 smithi133 ceph-mon[29610]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 186 KiB/s wr, 33 op/s 2024-05-09T08:32:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:28 smithi188 ceph-mon[34645]: pgmap v454: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 186 KiB/s wr, 33 op/s 2024-05-09T08:32:29.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23815 3.19 MB/sec warmup 45 sec latency 2684.599 ms 2024-05-09T08:32:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:29 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:32:30.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:29 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:32:30.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:29 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:32:30.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23815 3.12 MB/sec warmup 46 sec latency 3684.747 ms 2024-05-09T08:32:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:32:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:32:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:30 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:32:31.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:30 smithi160 ceph-mon[34070]: pgmap v455: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 175 KiB/s wr, 34 op/s 2024-05-09T08:32:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:32:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:32:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:30 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:32:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:30 smithi133 ceph-mon[29610]: pgmap v455: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 175 KiB/s wr, 34 op/s 2024-05-09T08:32:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:32:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:32:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:30 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:32:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:30 smithi188 ceph-mon[34645]: pgmap v455: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 175 KiB/s wr, 34 op/s 2024-05-09T08:32:31.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 23815 3.05 MB/sec warmup 47 sec latency 4684.899 ms 2024-05-09T08:32:32.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 25333 3.23 MB/sec warmup 48 sec latency 4854.196 ms 2024-05-09T08:32:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:32 smithi160 ceph-mon[34070]: pgmap v456: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-09T08:32:33.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:32 smithi133 ceph-mon[29610]: pgmap v456: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-09T08:32:33.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:32 smithi188 ceph-mon[34645]: pgmap v456: 97 pgs: 97 active+clean; 29 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-09T08:32:33.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27210 3.30 MB/sec warmup 49 sec latency 164.266 ms 2024-05-09T08:32:34.910 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27210 3.23 MB/sec warmup 50 sec latency 1164.412 ms 2024-05-09T08:32:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:34 smithi160 ceph-mon[34070]: pgmap v457: 97 pgs: 97 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 151 KiB/s wr, 31 op/s 2024-05-09T08:32:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:34 smithi133 ceph-mon[29610]: pgmap v457: 97 pgs: 97 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 151 KiB/s wr, 31 op/s 2024-05-09T08:32:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:34 smithi188 ceph-mon[34645]: pgmap v457: 97 pgs: 97 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 151 KiB/s wr, 31 op/s 2024-05-09T08:32:35.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27210 3.17 MB/sec warmup 51 sec latency 2164.549 ms 2024-05-09T08:32:36.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27304 3.12 MB/sec warmup 52 sec latency 2670.527 ms 2024-05-09T08:32:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:36 smithi160 ceph-mon[34070]: pgmap v458: 97 pgs: 97 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 191 KiB/s wr, 32 op/s 2024-05-09T08:32:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:36 smithi133 ceph-mon[29610]: pgmap v458: 97 pgs: 97 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 191 KiB/s wr, 32 op/s 2024-05-09T08:32:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:36 smithi188 ceph-mon[34645]: pgmap v458: 97 pgs: 97 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 191 KiB/s wr, 32 op/s 2024-05-09T08:32:37.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27365 3.07 MB/sec warmup 53 sec latency 684.481 ms 2024-05-09T08:32:38.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27365 3.01 MB/sec warmup 54 sec latency 1684.640 ms 2024-05-09T08:32:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:38 smithi160 ceph-mon[34070]: pgmap v459: 97 pgs: 97 active+clean; 31 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 372 KiB/s wr, 40 op/s 2024-05-09T08:32:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:38 smithi133 ceph-mon[29610]: pgmap v459: 97 pgs: 97 active+clean; 31 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 372 KiB/s wr, 40 op/s 2024-05-09T08:32:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:38 smithi188 ceph-mon[34645]: pgmap v459: 97 pgs: 97 active+clean; 31 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 372 KiB/s wr, 40 op/s 2024-05-09T08:32:39.912 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27365 2.96 MB/sec warmup 55 sec latency 2684.776 ms 2024-05-09T08:32:40.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27365 2.90 MB/sec warmup 56 sec latency 3684.929 ms 2024-05-09T08:32:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:40 smithi160 ceph-mon[34070]: pgmap v460: 97 pgs: 97 active+clean; 31 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 813 KiB/s wr, 54 op/s 2024-05-09T08:32:41.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:40 smithi133 ceph-mon[29610]: pgmap v460: 97 pgs: 97 active+clean; 31 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 813 KiB/s wr, 54 op/s 2024-05-09T08:32:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:40 smithi188 ceph-mon[34645]: pgmap v460: 97 pgs: 97 active+clean; 31 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 813 KiB/s wr, 54 op/s 2024-05-09T08:32:41.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 27365 2.85 MB/sec warmup 57 sec latency 4685.076 ms 2024-05-09T08:32:42.911 INFO:tasks.workunit.client.0.smithi133.stdout: 1 28834 3.00 MB/sec warmup 58 sec latency 4853.948 ms 2024-05-09T08:32:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:42 smithi160 ceph-mon[34070]: pgmap v461: 97 pgs: 97 active+clean; 30 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 934 KiB/s wr, 59 op/s 2024-05-09T08:32:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:42 smithi133 ceph-mon[29610]: pgmap v461: 97 pgs: 97 active+clean; 30 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 934 KiB/s wr, 59 op/s 2024-05-09T08:32:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:42 smithi188 ceph-mon[34645]: pgmap v461: 97 pgs: 97 active+clean; 30 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 934 KiB/s wr, 59 op/s 2024-05-09T08:32:43.912 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30753 3.06 MB/sec warmup 59 sec latency 74.131 ms 2024-05-09T08:32:44.912 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30753 3.01 MB/sec warmup 60 sec latency 1074.293 ms 2024-05-09T08:32:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:44 smithi160 ceph-mon[34070]: pgmap v462: 97 pgs: 97 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2024-05-09T08:32:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:44 smithi133 ceph-mon[29610]: pgmap v462: 97 pgs: 97 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2024-05-09T08:32:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:44 smithi188 ceph-mon[34645]: pgmap v462: 97 pgs: 97 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.0 MiB/s wr, 78 op/s 2024-05-09T08:32:45.912 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30753 2.96 MB/sec warmup 61 sec latency 2074.436 ms 2024-05-09T08:32:46.912 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30753 2.92 MB/sec warmup 62 sec latency 3074.568 ms 2024-05-09T08:32:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:46 smithi160 ceph-mon[34070]: pgmap v463: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 961 KiB/s wr, 60 op/s 2024-05-09T08:32:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:46 smithi133 ceph-mon[29610]: pgmap v463: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 961 KiB/s wr, 60 op/s 2024-05-09T08:32:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:46 smithi188 ceph-mon[34645]: pgmap v463: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 961 KiB/s wr, 60 op/s 2024-05-09T08:32:47.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30807 2.87 MB/sec warmup 63 sec latency 3233.382 ms 2024-05-09T08:32:48.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30807 2.83 MB/sec warmup 64 sec latency 1764.347 ms 2024-05-09T08:32:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:48 smithi160 ceph-mon[34070]: pgmap v464: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 918 KiB/s wr, 58 op/s 2024-05-09T08:32:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:48 smithi133 ceph-mon[29610]: pgmap v464: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 918 KiB/s wr, 58 op/s 2024-05-09T08:32:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:48 smithi188 ceph-mon[34645]: pgmap v464: 97 pgs: 97 active+clean; 31 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 918 KiB/s wr, 58 op/s 2024-05-09T08:32:49.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30807 2.78 MB/sec warmup 65 sec latency 2764.505 ms 2024-05-09T08:32:50.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30807 2.74 MB/sec warmup 66 sec latency 3764.596 ms 2024-05-09T08:32:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:50 smithi160 ceph-mon[34070]: pgmap v465: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 738 KiB/s wr, 50 op/s 2024-05-09T08:32:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:50 smithi133 ceph-mon[29610]: pgmap v465: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 738 KiB/s wr, 50 op/s 2024-05-09T08:32:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:50 smithi188 ceph-mon[34645]: pgmap v465: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 738 KiB/s wr, 50 op/s 2024-05-09T08:32:51.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30807 2.70 MB/sec warmup 67 sec latency 4764.741 ms 2024-05-09T08:32:52.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30847 2.66 MB/sec warmup 68 sec latency 4932.329 ms 2024-05-09T08:32:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:52 smithi160 ceph-mon[34070]: pgmap v466: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 290 KiB/s wr, 38 op/s 2024-05-09T08:32:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:52 smithi133 ceph-mon[29610]: pgmap v466: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 290 KiB/s wr, 38 op/s 2024-05-09T08:32:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:52 smithi188 ceph-mon[34645]: pgmap v466: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 290 KiB/s wr, 38 op/s 2024-05-09T08:32:53.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30847 2.62 MB/sec warmup 69 sec latency 1752.060 ms 2024-05-09T08:32:54.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30847 2.59 MB/sec warmup 70 sec latency 2752.206 ms 2024-05-09T08:32:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:54 smithi160 ceph-mon[34070]: pgmap v467: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 169 KiB/s wr, 33 op/s 2024-05-09T08:32:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:54 smithi133 ceph-mon[29610]: pgmap v467: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 169 KiB/s wr, 33 op/s 2024-05-09T08:32:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:54 smithi188 ceph-mon[34645]: pgmap v467: 97 pgs: 97 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 169 KiB/s wr, 33 op/s 2024-05-09T08:32:55.913 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30847 2.55 MB/sec warmup 71 sec latency 3752.350 ms 2024-05-09T08:32:56.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30847 2.52 MB/sec warmup 72 sec latency 4752.496 ms 2024-05-09T08:32:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:56 smithi160 ceph-mon[34070]: pgmap v468: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-09T08:32:57.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:56 smithi133 ceph-mon[29610]: pgmap v468: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-09T08:32:57.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:56 smithi188 ceph-mon[34645]: pgmap v468: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-09T08:32:57.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30908 2.49 MB/sec warmup 73 sec latency 4917.045 ms 2024-05-09T08:32:58.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30908 2.46 MB/sec warmup 74 sec latency 1709.808 ms 2024-05-09T08:32:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:32:58 smithi160 ceph-mon[34070]: pgmap v469: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-05-09T08:32:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:32:58 smithi133 ceph-mon[29610]: pgmap v469: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-05-09T08:32:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:32:58 smithi188 ceph-mon[34645]: pgmap v469: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-05-09T08:32:59.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30908 2.42 MB/sec warmup 75 sec latency 2709.941 ms 2024-05-09T08:33:00.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30908 2.39 MB/sec warmup 76 sec latency 3710.046 ms 2024-05-09T08:33:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:00 smithi160 ceph-mon[34070]: pgmap v470: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-09T08:33:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:00 smithi133 ceph-mon[29610]: pgmap v470: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-09T08:33:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:00 smithi188 ceph-mon[34645]: pgmap v470: 97 pgs: 97 active+clean; 31 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-05-09T08:33:01.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 30908 2.36 MB/sec warmup 77 sec latency 4710.182 ms 2024-05-09T08:33:02.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 32338 2.48 MB/sec warmup 78 sec latency 4876.316 ms 2024-05-09T08:33:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:02 smithi160 ceph-mon[34070]: pgmap v471: 97 pgs: 97 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-09T08:33:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:02 smithi133 ceph-mon[29610]: pgmap v471: 97 pgs: 97 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-09T08:33:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:02 smithi188 ceph-mon[34645]: pgmap v471: 97 pgs: 97 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-09T08:33:03.914 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34300 2.53 MB/sec warmup 79 sec latency 27.240 ms 2024-05-09T08:33:04.915 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34300 2.50 MB/sec warmup 80 sec latency 1011.236 ms 2024-05-09T08:33:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:04 smithi160 ceph-mon[34070]: pgmap v472: 97 pgs: 97 active+clean; 34 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 593 KiB/s wr, 36 op/s 2024-05-09T08:33:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:04 smithi133 ceph-mon[29610]: pgmap v472: 97 pgs: 97 active+clean; 34 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 593 KiB/s wr, 36 op/s 2024-05-09T08:33:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:04 smithi188 ceph-mon[34645]: pgmap v472: 97 pgs: 97 active+clean; 34 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 593 KiB/s wr, 36 op/s 2024-05-09T08:33:05.915 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34300 2.47 MB/sec warmup 81 sec latency 2011.398 ms 2024-05-09T08:33:06.915 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34300 2.44 MB/sec warmup 82 sec latency 3011.611 ms 2024-05-09T08:33:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:06 smithi160 ceph-mon[34070]: pgmap v473: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 633 KiB/s wr, 40 op/s 2024-05-09T08:33:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:06 smithi133 ceph-mon[29610]: pgmap v473: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 633 KiB/s wr, 40 op/s 2024-05-09T08:33:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:06 smithi188 ceph-mon[34645]: pgmap v473: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 633 KiB/s wr, 40 op/s 2024-05-09T08:33:07.915 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34354 2.41 MB/sec warmup 83 sec latency 3166.994 ms 2024-05-09T08:33:08.915 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34354 2.38 MB/sec warmup 84 sec latency 1759.847 ms 2024-05-09T08:33:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:08 smithi160 ceph-mon[34070]: pgmap v474: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 670 KiB/s wr, 43 op/s 2024-05-09T08:33:09.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:08 smithi133 ceph-mon[29610]: pgmap v474: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 670 KiB/s wr, 43 op/s 2024-05-09T08:33:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:08 smithi188 ceph-mon[34645]: pgmap v474: 97 pgs: 97 active+clean; 34 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 670 KiB/s wr, 43 op/s 2024-05-09T08:33:09.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34354 2.35 MB/sec warmup 85 sec latency 2759.997 ms 2024-05-09T08:33:10.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34354 2.33 MB/sec warmup 86 sec latency 3760.150 ms 2024-05-09T08:33:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:10 smithi160 ceph-mon[34070]: pgmap v475: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 663 KiB/s wr, 44 op/s 2024-05-09T08:33:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:10 smithi133 ceph-mon[29610]: pgmap v475: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 663 KiB/s wr, 44 op/s 2024-05-09T08:33:11.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:10 smithi188 ceph-mon[34645]: pgmap v475: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 663 KiB/s wr, 44 op/s 2024-05-09T08:33:11.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34354 2.30 MB/sec warmup 87 sec latency 4760.300 ms 2024-05-09T08:33:12.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34394 2.28 MB/sec warmup 88 sec latency 4926.673 ms 2024-05-09T08:33:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:12 smithi160 ceph-mon[34070]: pgmap v476: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 655 KiB/s wr, 44 op/s 2024-05-09T08:33:13.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:12 smithi133 ceph-mon[29610]: pgmap v476: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 655 KiB/s wr, 44 op/s 2024-05-09T08:33:13.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:12 smithi188 ceph-mon[34645]: pgmap v476: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 655 KiB/s wr, 44 op/s 2024-05-09T08:33:13.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34394 2.25 MB/sec warmup 89 sec latency 1751.608 ms 2024-05-09T08:33:14.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34394 2.22 MB/sec warmup 90 sec latency 2751.704 ms 2024-05-09T08:33:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:14 smithi160 ceph-mon[34070]: pgmap v477: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 665 KiB/s wr, 46 op/s 2024-05-09T08:33:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:14 smithi133 ceph-mon[29610]: pgmap v477: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 665 KiB/s wr, 46 op/s 2024-05-09T08:33:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:14 smithi188 ceph-mon[34645]: pgmap v477: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 665 KiB/s wr, 46 op/s 2024-05-09T08:33:15.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34394 2.20 MB/sec warmup 91 sec latency 3751.877 ms 2024-05-09T08:33:16.916 INFO:tasks.workunit.client.0.smithi133.stdout: 1 34394 2.18 MB/sec warmup 92 sec latency 4752.044 ms 2024-05-09T08:33:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:16 smithi160 ceph-mon[34070]: pgmap v478: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-05-09T08:33:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:16 smithi133 ceph-mon[29610]: pgmap v478: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-05-09T08:33:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:16 smithi188 ceph-mon[34645]: pgmap v478: 97 pgs: 97 active+clean; 32 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 95 KiB/s wr, 20 op/s 2024-05-09T08:33:17.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 35711 2.28 MB/sec warmup 93 sec latency 4915.665 ms 2024-05-09T08:33:18.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37284 2.31 MB/sec warmup 94 sec latency 15.317 ms 2024-05-09T08:33:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:18 smithi160 ceph-mon[34070]: pgmap v479: 97 pgs: 97 active+clean; 33 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 31 op/s 2024-05-09T08:33:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:18 smithi133 ceph-mon[29610]: pgmap v479: 97 pgs: 97 active+clean; 33 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 31 op/s 2024-05-09T08:33:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:18 smithi188 ceph-mon[34645]: pgmap v479: 97 pgs: 97 active+clean; 33 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 31 op/s 2024-05-09T08:33:19.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37843 2.31 MB/sec warmup 95 sec latency 790.338 ms 2024-05-09T08:33:20.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37843 2.28 MB/sec warmup 96 sec latency 1790.489 ms 2024-05-09T08:33:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:20 smithi160 ceph-mon[34070]: pgmap v480: 97 pgs: 97 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 336 KiB/s wr, 35 op/s 2024-05-09T08:33:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:20 smithi133 ceph-mon[29610]: pgmap v480: 97 pgs: 97 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 336 KiB/s wr, 35 op/s 2024-05-09T08:33:21.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:20 smithi188 ceph-mon[34645]: pgmap v480: 97 pgs: 97 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 336 KiB/s wr, 35 op/s 2024-05-09T08:33:21.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37843 2.26 MB/sec warmup 97 sec latency 2790.636 ms 2024-05-09T08:33:22.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37937 2.24 MB/sec warmup 98 sec latency 2947.530 ms 2024-05-09T08:33:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:22 smithi160 ceph-mon[34070]: pgmap v481: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 505 KiB/s wr, 45 op/s 2024-05-09T08:33:23.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:22 smithi133 ceph-mon[29610]: pgmap v481: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 505 KiB/s wr, 45 op/s 2024-05-09T08:33:23.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:22 smithi188 ceph-mon[34645]: pgmap v481: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 505 KiB/s wr, 45 op/s 2024-05-09T08:33:23.917 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37937 2.22 MB/sec warmup 99 sec latency 1652.401 ms 2024-05-09T08:33:24.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37937 2.19 MB/sec warmup 100 sec latency 2652.548 ms 2024-05-09T08:33:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:24 smithi160 ceph-mon[34070]: pgmap v482: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 538 KiB/s wr, 51 op/s 2024-05-09T08:33:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:24 smithi133 ceph-mon[29610]: pgmap v482: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 538 KiB/s wr, 51 op/s 2024-05-09T08:33:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:24 smithi188 ceph-mon[34645]: pgmap v482: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 538 KiB/s wr, 51 op/s 2024-05-09T08:33:25.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37937 2.17 MB/sec warmup 101 sec latency 3652.746 ms 2024-05-09T08:33:26.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 37937 2.15 MB/sec warmup 102 sec latency 4652.841 ms 2024-05-09T08:33:27.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 39300 2.24 MB/sec warmup 103 sec latency 4813.875 ms 2024-05-09T08:33:28.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:27 smithi160 ceph-mon[34070]: pgmap v483: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 532 KiB/s wr, 50 op/s 2024-05-09T08:33:28.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:27 smithi133 ceph-mon[29610]: pgmap v483: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 532 KiB/s wr, 50 op/s 2024-05-09T08:33:28.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:27 smithi188 ceph-mon[34645]: pgmap v483: 97 pgs: 97 active+clean; 33 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 532 KiB/s wr, 50 op/s 2024-05-09T08:33:28.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 41198 2.28 MB/sec warmup 104 sec latency 11.294 ms 2024-05-09T08:33:29.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 42396 2.34 MB/sec warmup 105 sec latency 23.176 ms 2024-05-09T08:33:30.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:29 smithi160 ceph-mon[34070]: pgmap v484: 97 pgs: 97 active+clean; 34 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 623 KiB/s wr, 64 op/s 2024-05-09T08:33:30.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:29 smithi133 ceph-mon[29610]: pgmap v484: 97 pgs: 97 active+clean; 34 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 623 KiB/s wr, 64 op/s 2024-05-09T08:33:30.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:29 smithi188 ceph-mon[34645]: pgmap v484: 97 pgs: 97 active+clean; 34 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 623 KiB/s wr, 64 op/s 2024-05-09T08:33:30.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 43500 2.37 MB/sec warmup 106 sec latency 18.257 ms 2024-05-09T08:33:31.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 45199 2.42 MB/sec warmup 107 sec latency 19.484 ms 2024-05-09T08:33:32.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:31 smithi160 ceph-mon[34070]: pgmap v485: 97 pgs: 97 active+clean; 35 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-05-09T08:33:32.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:31 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:33:32.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:31 smithi133 ceph-mon[29610]: pgmap v485: 97 pgs: 97 active+clean; 35 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-05-09T08:33:32.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:31 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:33:32.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:31 smithi188 ceph-mon[34645]: pgmap v485: 97 pgs: 97 active+clean; 35 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-05-09T08:33:32.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:31 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:33:32.918 INFO:tasks.workunit.client.0.smithi133.stdout: 1 46882 2.50 MB/sec warmup 108 sec latency 13.962 ms 2024-05-09T08:33:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:32 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:33:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:32 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:33:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:32 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:33:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:33:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:33:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:32 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:33:33.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:32 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:33:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:32 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:33:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:32 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:33:33.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 48577 2.54 MB/sec warmup 109 sec latency 23.797 ms 2024-05-09T08:33:34.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:34 smithi160 ceph-mon[34070]: pgmap v486: 97 pgs: 97 active+clean; 36 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 435 KiB/s wr, 65 op/s 2024-05-09T08:33:34.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:34 smithi133 ceph-mon[29610]: pgmap v486: 97 pgs: 97 active+clean; 36 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 435 KiB/s wr, 65 op/s 2024-05-09T08:33:34.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:34 smithi188 ceph-mon[34645]: pgmap v486: 97 pgs: 97 active+clean; 36 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 435 KiB/s wr, 65 op/s 2024-05-09T08:33:34.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 49971 2.62 MB/sec warmup 110 sec latency 19.749 ms 2024-05-09T08:33:35.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 51358 2.65 MB/sec warmup 111 sec latency 12.263 ms 2024-05-09T08:33:36.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:36 smithi160 ceph-mon[34070]: pgmap v487: 97 pgs: 97 active+clean; 38 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 540 KiB/s wr, 103 op/s 2024-05-09T08:33:36.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:36 smithi133 ceph-mon[29610]: pgmap v487: 97 pgs: 97 active+clean; 38 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 540 KiB/s wr, 103 op/s 2024-05-09T08:33:36.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:36 smithi188 ceph-mon[34645]: pgmap v487: 97 pgs: 97 active+clean; 38 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 540 KiB/s wr, 103 op/s 2024-05-09T08:33:36.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 52687 2.69 MB/sec warmup 112 sec latency 22.636 ms 2024-05-09T08:33:37.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 54120 2.73 MB/sec warmup 113 sec latency 12.478 ms 2024-05-09T08:33:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:38 smithi160 ceph-mon[34070]: pgmap v488: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 553 KiB/s wr, 103 op/s 2024-05-09T08:33:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:38 smithi133 ceph-mon[29610]: pgmap v488: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 553 KiB/s wr, 103 op/s 2024-05-09T08:33:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:38 smithi188 ceph-mon[34645]: pgmap v488: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 553 KiB/s wr, 103 op/s 2024-05-09T08:33:38.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 55715 2.77 MB/sec warmup 114 sec latency 20.249 ms 2024-05-09T08:33:39.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 57151 2.84 MB/sec warmup 115 sec latency 20.412 ms 2024-05-09T08:33:40.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:40 smithi160 ceph-mon[34070]: pgmap v489: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 616 KiB/s wr, 111 op/s 2024-05-09T08:33:40.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:40 smithi133 ceph-mon[29610]: pgmap v489: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 616 KiB/s wr, 111 op/s 2024-05-09T08:33:40.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:40 smithi188 ceph-mon[34645]: pgmap v489: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 616 KiB/s wr, 111 op/s 2024-05-09T08:33:40.919 INFO:tasks.workunit.client.0.smithi133.stdout: 1 58696 2.87 MB/sec warmup 116 sec latency 15.534 ms 2024-05-09T08:33:41.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 60140 2.92 MB/sec warmup 117 sec latency 28.303 ms 2024-05-09T08:33:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:42 smithi160 ceph-mon[34070]: pgmap v490: 97 pgs: 97 active+clean; 42 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 785 KiB/s wr, 141 op/s 2024-05-09T08:33:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:42 smithi133 ceph-mon[29610]: pgmap v490: 97 pgs: 97 active+clean; 42 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 785 KiB/s wr, 141 op/s 2024-05-09T08:33:42.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:42 smithi188 ceph-mon[34645]: pgmap v490: 97 pgs: 97 active+clean; 42 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 785 KiB/s wr, 141 op/s 2024-05-09T08:33:42.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 61378 2.95 MB/sec warmup 118 sec latency 11.488 ms 2024-05-09T08:33:43.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 62881 2.97 MB/sec warmup 119 sec latency 19.447 ms 2024-05-09T08:33:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:44 smithi160 ceph-mon[34070]: pgmap v491: 97 pgs: 97 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 846 KiB/s wr, 156 op/s 2024-05-09T08:33:44.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:44 smithi133 ceph-mon[29610]: pgmap v491: 97 pgs: 97 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 846 KiB/s wr, 156 op/s 2024-05-09T08:33:44.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:44 smithi188 ceph-mon[34645]: pgmap v491: 97 pgs: 97 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 846 KiB/s wr, 156 op/s 2024-05-09T08:33:45.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 66248 6.99 MB/sec execute 1 sec latency 18.620 ms 2024-05-09T08:33:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:46 smithi160 ceph-mon[34070]: pgmap v492: 97 pgs: 97 active+clean; 44 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 912 KiB/s wr, 157 op/s 2024-05-09T08:33:46.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:46 smithi133 ceph-mon[29610]: pgmap v492: 97 pgs: 97 active+clean; 44 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 912 KiB/s wr, 157 op/s 2024-05-09T08:33:46.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:46 smithi188 ceph-mon[34645]: pgmap v492: 97 pgs: 97 active+clean; 44 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 912 KiB/s wr, 157 op/s 2024-05-09T08:33:46.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 67641 9.40 MB/sec execute 2 sec latency 18.822 ms 2024-05-09T08:33:47.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 68872 8.06 MB/sec execute 3 sec latency 22.395 ms 2024-05-09T08:33:48.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:48 smithi160 ceph-mon[34070]: pgmap v493: 97 pgs: 97 active+clean; 46 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 859 KiB/s wr, 148 op/s 2024-05-09T08:33:48.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:48 smithi133 ceph-mon[29610]: pgmap v493: 97 pgs: 97 active+clean; 46 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 859 KiB/s wr, 148 op/s 2024-05-09T08:33:48.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:48 smithi188 ceph-mon[34645]: pgmap v493: 97 pgs: 97 active+clean; 46 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 859 KiB/s wr, 148 op/s 2024-05-09T08:33:48.920 INFO:tasks.workunit.client.0.smithi133.stdout: 1 70185 7.53 MB/sec execute 4 sec latency 27.171 ms 2024-05-09T08:33:49.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 71695 7.71 MB/sec execute 5 sec latency 12.410 ms 2024-05-09T08:33:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:50 smithi160 ceph-mon[34070]: pgmap v494: 97 pgs: 97 active+clean; 47 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 933 KiB/s wr, 151 op/s 2024-05-09T08:33:50.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:50 smithi133 ceph-mon[29610]: pgmap v494: 97 pgs: 97 active+clean; 47 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 933 KiB/s wr, 151 op/s 2024-05-09T08:33:50.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:50 smithi188 ceph-mon[34645]: pgmap v494: 97 pgs: 97 active+clean; 47 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 933 KiB/s wr, 151 op/s 2024-05-09T08:33:50.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 73427 7.57 MB/sec execute 6 sec latency 20.541 ms 2024-05-09T08:33:51.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 74816 8.16 MB/sec execute 7 sec latency 23.037 ms 2024-05-09T08:33:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:52 smithi160 ceph-mon[34070]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 951 KiB/s wr, 153 op/s 2024-05-09T08:33:52.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:52 smithi133 ceph-mon[29610]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 951 KiB/s wr, 153 op/s 2024-05-09T08:33:52.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:52 smithi188 ceph-mon[34645]: pgmap v495: 97 pgs: 97 active+clean; 47 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 951 KiB/s wr, 153 op/s 2024-05-09T08:33:52.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 76164 7.83 MB/sec execute 8 sec latency 14.688 ms 2024-05-09T08:33:53.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 77421 7.63 MB/sec execute 9 sec latency 36.485 ms 2024-05-09T08:33:54.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:54 smithi160 ceph-mon[34070]: pgmap v496: 97 pgs: 97 active+clean; 49 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 950 KiB/s wr, 151 op/s 2024-05-09T08:33:54.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:54 smithi133 ceph-mon[29610]: pgmap v496: 97 pgs: 97 active+clean; 49 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 950 KiB/s wr, 151 op/s 2024-05-09T08:33:54.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:54 smithi188 ceph-mon[34645]: pgmap v496: 97 pgs: 97 active+clean; 49 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 950 KiB/s wr, 151 op/s 2024-05-09T08:33:54.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 78688 7.65 MB/sec execute 10 sec latency 18.370 ms 2024-05-09T08:33:55.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 80465 7.59 MB/sec execute 11 sec latency 19.630 ms 2024-05-09T08:33:56.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:56 smithi133 ceph-mon[29610]: pgmap v497: 97 pgs: 97 active+clean; 51 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-09T08:33:56.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:56 smithi188 ceph-mon[34645]: pgmap v497: 97 pgs: 97 active+clean; 51 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-09T08:33:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:56 smithi160 ceph-mon[34070]: pgmap v497: 97 pgs: 97 active+clean; 51 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-05-09T08:33:56.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 81719 7.85 MB/sec execute 12 sec latency 22.173 ms 2024-05-09T08:33:57.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 82814 7.52 MB/sec execute 13 sec latency 18.636 ms 2024-05-09T08:33:58.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:33:58 smithi133 ceph-mon[29610]: pgmap v498: 97 pgs: 97 active+clean; 51 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 960 KiB/s wr, 153 op/s 2024-05-09T08:33:58.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:33:58 smithi188 ceph-mon[34645]: pgmap v498: 97 pgs: 97 active+clean; 51 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 960 KiB/s wr, 153 op/s 2024-05-09T08:33:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:33:58 smithi160 ceph-mon[34070]: pgmap v498: 97 pgs: 97 active+clean; 51 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 818 KiB/s rd, 960 KiB/s wr, 153 op/s 2024-05-09T08:33:58.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 84217 7.40 MB/sec execute 14 sec latency 37.297 ms 2024-05-09T08:33:59.921 INFO:tasks.workunit.client.0.smithi133.stdout: 1 85627 7.65 MB/sec execute 15 sec latency 21.021 ms 2024-05-09T08:34:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:00 smithi160 ceph-mon[34070]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1012 KiB/s wr, 156 op/s 2024-05-09T08:34:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:00 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:34:00.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 87468 7.60 MB/sec execute 16 sec latency 15.341 ms 2024-05-09T08:34:00.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:00 smithi133 ceph-mon[29610]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1012 KiB/s wr, 156 op/s 2024-05-09T08:34:00.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:00 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:34:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:00 smithi188 ceph-mon[34645]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1012 KiB/s wr, 156 op/s 2024-05-09T08:34:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:00 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:34:01.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 88534 7.60 MB/sec execute 17 sec latency 22.161 ms 2024-05-09T08:34:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:02 smithi160 ceph-mon[34070]: pgmap v500: 97 pgs: 97 active+clean; 55 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-05-09T08:34:02.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 90126 7.72 MB/sec execute 18 sec latency 9.791 ms 2024-05-09T08:34:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:02 smithi133 ceph-mon[29610]: pgmap v500: 97 pgs: 97 active+clean; 55 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-05-09T08:34:02.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:02 smithi188 ceph-mon[34645]: pgmap v500: 97 pgs: 97 active+clean; 55 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.0 MiB/s wr, 158 op/s 2024-05-09T08:34:03.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 91507 7.63 MB/sec execute 19 sec latency 22.413 ms 2024-05-09T08:34:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:04 smithi160 ceph-mon[34070]: pgmap v501: 97 pgs: 97 active+clean; 55 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-05-09T08:34:04.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 93177 7.69 MB/sec execute 20 sec latency 22.535 ms 2024-05-09T08:34:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:04 smithi133 ceph-mon[29610]: pgmap v501: 97 pgs: 97 active+clean; 55 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-05-09T08:34:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:04 smithi188 ceph-mon[34645]: pgmap v501: 97 pgs: 97 active+clean; 55 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-05-09T08:34:05.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 94837 7.66 MB/sec execute 21 sec latency 19.446 ms 2024-05-09T08:34:06.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:06 smithi160 ceph-mon[34070]: pgmap v502: 97 pgs: 97 active+clean; 58 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-09T08:34:06.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 96538 7.84 MB/sec execute 22 sec latency 21.701 ms 2024-05-09T08:34:06.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:06 smithi133 ceph-mon[29610]: pgmap v502: 97 pgs: 97 active+clean; 58 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-09T08:34:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:06 smithi188 ceph-mon[34645]: pgmap v502: 97 pgs: 97 active+clean; 58 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-05-09T08:34:07.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 98308 7.82 MB/sec execute 23 sec latency 20.706 ms 2024-05-09T08:34:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:08 smithi160 ceph-mon[34070]: pgmap v503: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-09T08:34:08.922 INFO:tasks.workunit.client.0.smithi133.stdout: 1 100026 7.97 MB/sec execute 24 sec latency 17.867 ms 2024-05-09T08:34:08.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:08 smithi133 ceph-mon[29610]: pgmap v503: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-09T08:34:08.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:08 smithi188 ceph-mon[34645]: pgmap v503: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-09T08:34:09.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 101801 7.94 MB/sec execute 25 sec latency 38.402 ms 2024-05-09T08:34:10.734 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:10 smithi188 ceph-mon[34645]: pgmap v504: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-09T08:34:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:10 smithi160 ceph-mon[34070]: pgmap v504: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-09T08:34:10.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 103354 8.09 MB/sec execute 26 sec latency 17.547 ms 2024-05-09T08:34:10.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:10 smithi133 ceph-mon[29610]: pgmap v504: 97 pgs: 97 active+clean; 59 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-09T08:34:11.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 105300 8.05 MB/sec execute 27 sec latency 17.966 ms 2024-05-09T08:34:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:12 smithi160 ceph-mon[34070]: pgmap v505: 97 pgs: 97 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-05-09T08:34:12.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 106739 8.19 MB/sec execute 28 sec latency 23.718 ms 2024-05-09T08:34:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:12 smithi133 ceph-mon[29610]: pgmap v505: 97 pgs: 97 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-05-09T08:34:12.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:12 smithi188 ceph-mon[34645]: pgmap v505: 97 pgs: 97 active+clean; 61 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-05-09T08:34:13.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 108768 8.15 MB/sec execute 29 sec latency 11.860 ms 2024-05-09T08:34:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:14 smithi160 ceph-mon[34070]: pgmap v506: 97 pgs: 97 active+clean; 64 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:34:14.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 109840 8.13 MB/sec execute 30 sec latency 123.731 ms 2024-05-09T08:34:14.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:14 smithi133 ceph-mon[29610]: pgmap v506: 97 pgs: 97 active+clean; 64 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:34:14.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:14 smithi188 ceph-mon[34645]: pgmap v506: 97 pgs: 97 active+clean; 64 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:34:15.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 111488 8.18 MB/sec execute 31 sec latency 17.351 ms 2024-05-09T08:34:16.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:16 smithi160 ceph-mon[34070]: pgmap v507: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-09T08:34:16.923 INFO:tasks.workunit.client.0.smithi133.stdout: 1 112978 8.16 MB/sec execute 32 sec latency 19.878 ms 2024-05-09T08:34:16.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:16 smithi133 ceph-mon[29610]: pgmap v507: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-09T08:34:16.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:16 smithi188 ceph-mon[34645]: pgmap v507: 97 pgs: 97 active+clean; 62 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.2 MiB/s wr, 206 op/s 2024-05-09T08:34:17.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 114657 8.17 MB/sec execute 33 sec latency 10.511 ms 2024-05-09T08:34:18.730 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:18 smithi133 ceph-mon[29610]: pgmap v508: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:18 smithi160 ceph-mon[34070]: pgmap v508: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:18.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 116151 8.11 MB/sec execute 34 sec latency 20.989 ms 2024-05-09T08:34:18.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:18 smithi188 ceph-mon[34645]: pgmap v508: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:19.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 117754 8.20 MB/sec execute 35 sec latency 42.159 ms 2024-05-09T08:34:20.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:20 smithi160 ceph-mon[34070]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:20.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 119546 8.17 MB/sec execute 36 sec latency 21.300 ms 2024-05-09T08:34:20.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:20 smithi133 ceph-mon[29610]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:20.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:20 smithi188 ceph-mon[34645]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:21.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 121169 8.27 MB/sec execute 37 sec latency 17.687 ms 2024-05-09T08:34:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:22 smithi160 ceph-mon[34070]: pgmap v510: 97 pgs: 97 active+clean; 66 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2024-05-09T08:34:22.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 123073 8.24 MB/sec execute 38 sec latency 17.982 ms 2024-05-09T08:34:22.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:22 smithi133 ceph-mon[29610]: pgmap v510: 97 pgs: 97 active+clean; 66 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2024-05-09T08:34:22.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:22 smithi188 ceph-mon[34645]: pgmap v510: 97 pgs: 97 active+clean; 66 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 200 op/s 2024-05-09T08:34:23.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 124620 8.34 MB/sec execute 39 sec latency 18.222 ms 2024-05-09T08:34:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:24 smithi160 ceph-mon[34070]: pgmap v511: 97 pgs: 97 active+clean; 67 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-09T08:34:24.924 INFO:tasks.workunit.client.0.smithi133.stdout: 1 126514 8.30 MB/sec execute 40 sec latency 23.234 ms 2024-05-09T08:34:24.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:24 smithi133 ceph-mon[29610]: pgmap v511: 97 pgs: 97 active+clean; 67 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-09T08:34:24.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:24 smithi188 ceph-mon[34645]: pgmap v511: 97 pgs: 97 active+clean; 67 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-09T08:34:25.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 127537 8.28 MB/sec execute 41 sec latency 206.823 ms 2024-05-09T08:34:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:26 smithi160 ceph-mon[34070]: pgmap v512: 97 pgs: 97 active+clean; 68 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-09T08:34:26.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 129162 8.32 MB/sec execute 42 sec latency 13.221 ms 2024-05-09T08:34:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:26 smithi133 ceph-mon[29610]: pgmap v512: 97 pgs: 97 active+clean; 68 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-09T08:34:26.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:26 smithi188 ceph-mon[34645]: pgmap v512: 97 pgs: 97 active+clean; 68 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-05-09T08:34:27.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 130695 8.30 MB/sec execute 43 sec latency 17.650 ms 2024-05-09T08:34:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:28 smithi160 ceph-mon[34070]: pgmap v513: 97 pgs: 97 active+clean; 68 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:28.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 132250 8.28 MB/sec execute 44 sec latency 21.657 ms 2024-05-09T08:34:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:28 smithi133 ceph-mon[29610]: pgmap v513: 97 pgs: 97 active+clean; 68 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:28.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:28 smithi188 ceph-mon[34645]: pgmap v513: 97 pgs: 97 active+clean; 68 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 218 op/s 2024-05-09T08:34:29.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 133756 8.25 MB/sec execute 45 sec latency 61.145 ms 2024-05-09T08:34:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:30 smithi160 ceph-mon[34070]: pgmap v514: 97 pgs: 97 active+clean; 68 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-09T08:34:30.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 135140 8.31 MB/sec execute 46 sec latency 25.292 ms 2024-05-09T08:34:30.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:30 smithi133 ceph-mon[29610]: pgmap v514: 97 pgs: 97 active+clean; 68 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-09T08:34:30.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:30 smithi188 ceph-mon[34645]: pgmap v514: 97 pgs: 97 active+clean; 68 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-05-09T08:34:31.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 136646 8.25 MB/sec execute 47 sec latency 14.456 ms 2024-05-09T08:34:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:32 smithi160 ceph-mon[34070]: pgmap v515: 97 pgs: 97 active+clean; 70 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:32.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 138014 8.25 MB/sec execute 48 sec latency 22.857 ms 2024-05-09T08:34:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:32 smithi133 ceph-mon[29610]: pgmap v515: 97 pgs: 97 active+clean; 70 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:32.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:32 smithi188 ceph-mon[34645]: pgmap v515: 97 pgs: 97 active+clean; 70 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:33.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:33 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:34:33.925 INFO:tasks.workunit.client.0.smithi133.stdout: 1 139475 8.24 MB/sec execute 49 sec latency 14.618 ms 2024-05-09T08:34:33.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:33 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:34:33.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:33 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:34:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:34 smithi160 ceph-mon[34070]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T08:34:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:34 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:34:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:34 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:34:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:34 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:34:34.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 140943 8.19 MB/sec execute 50 sec latency 65.419 ms 2024-05-09T08:34:34.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:34 smithi133 ceph-mon[29610]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T08:34:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:34 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:34:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:34 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:34:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:34 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:34:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:34 smithi188 ceph-mon[34645]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-09T08:34:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:34 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:34:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:34 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:34:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:34 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:34:35.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 142380 8.25 MB/sec execute 51 sec latency 32.761 ms 2024-05-09T08:34:36.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:36 smithi160 ceph-mon[34070]: pgmap v517: 97 pgs: 97 active+clean; 71 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:34:36.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 144223 8.22 MB/sec execute 52 sec latency 18.345 ms 2024-05-09T08:34:36.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:36 smithi133 ceph-mon[29610]: pgmap v517: 97 pgs: 97 active+clean; 71 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:34:36.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:36 smithi188 ceph-mon[34645]: pgmap v517: 97 pgs: 97 active+clean; 71 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:34:37.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 145236 8.21 MB/sec execute 53 sec latency 28.083 ms 2024-05-09T08:34:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:38 smithi160 ceph-mon[34070]: pgmap v518: 97 pgs: 97 active+clean; 74 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:38.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 146428 8.16 MB/sec execute 54 sec latency 15.442 ms 2024-05-09T08:34:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:38 smithi133 ceph-mon[29610]: pgmap v518: 97 pgs: 97 active+clean; 74 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:38.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:38 smithi188 ceph-mon[34645]: pgmap v518: 97 pgs: 97 active+clean; 74 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-05-09T08:34:39.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 147963 8.14 MB/sec execute 55 sec latency 58.726 ms 2024-05-09T08:34:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:40 smithi160 ceph-mon[34070]: pgmap v519: 97 pgs: 97 active+clean; 74 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:34:40.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 149272 8.19 MB/sec execute 56 sec latency 242.915 ms 2024-05-09T08:34:40.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:40 smithi133 ceph-mon[29610]: pgmap v519: 97 pgs: 97 active+clean; 74 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:34:40.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:40 smithi188 ceph-mon[34645]: pgmap v519: 97 pgs: 97 active+clean; 74 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:34:41.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 150529 8.14 MB/sec execute 57 sec latency 79.457 ms 2024-05-09T08:34:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:42 smithi160 ceph-mon[34070]: pgmap v520: 97 pgs: 97 active+clean; 74 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:34:42.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 151505 8.05 MB/sec execute 58 sec latency 280.053 ms 2024-05-09T08:34:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:42 smithi133 ceph-mon[29610]: pgmap v520: 97 pgs: 97 active+clean; 74 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:34:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:42 smithi188 ceph-mon[34645]: pgmap v520: 97 pgs: 97 active+clean; 74 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:34:43.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 152725 8.08 MB/sec execute 59 sec latency 587.763 ms 2024-05-09T08:34:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:44 smithi160 ceph-mon[34070]: pgmap v521: 97 pgs: 97 active+clean; 75 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:34:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:44 smithi133 ceph-mon[29610]: pgmap v521: 97 pgs: 97 active+clean; 75 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:34:44.926 INFO:tasks.workunit.client.0.smithi133.stdout: 1 153952 8.05 MB/sec execute 60 sec latency 27.803 ms 2024-05-09T08:34:44.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:44 smithi188 ceph-mon[34645]: pgmap v521: 97 pgs: 97 active+clean; 75 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:34:45.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 155156 7.97 MB/sec execute 61 sec latency 51.273 ms 2024-05-09T08:34:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:46 smithi160 ceph-mon[34070]: pgmap v522: 97 pgs: 97 active+clean; 76 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:46.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:46 smithi133 ceph-mon[29610]: pgmap v522: 97 pgs: 97 active+clean; 76 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:46.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 156681 8.02 MB/sec execute 62 sec latency 16.322 ms 2024-05-09T08:34:46.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:46 smithi188 ceph-mon[34645]: pgmap v522: 97 pgs: 97 active+clean; 76 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:34:47.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 158397 8.00 MB/sec execute 63 sec latency 177.692 ms 2024-05-09T08:34:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:48 smithi160 ceph-mon[34070]: pgmap v523: 97 pgs: 97 active+clean; 76 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:34:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:48 smithi133 ceph-mon[29610]: pgmap v523: 97 pgs: 97 active+clean; 76 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:34:48.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 158695 7.89 MB/sec execute 64 sec latency 521.341 ms 2024-05-09T08:34:48.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:48 smithi188 ceph-mon[34645]: pgmap v523: 97 pgs: 97 active+clean; 76 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:34:49.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 160251 7.94 MB/sec execute 65 sec latency 17.389 ms 2024-05-09T08:34:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:50 smithi160 ceph-mon[34070]: pgmap v524: 97 pgs: 97 active+clean; 77 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:34:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:50 smithi133 ceph-mon[29610]: pgmap v524: 97 pgs: 97 active+clean; 77 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:34:50.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 162035 7.93 MB/sec execute 66 sec latency 52.789 ms 2024-05-09T08:34:50.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:50 smithi188 ceph-mon[34645]: pgmap v524: 97 pgs: 97 active+clean; 77 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:34:51.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 163456 7.99 MB/sec execute 67 sec latency 22.356 ms 2024-05-09T08:34:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:52 smithi160 ceph-mon[34070]: pgmap v525: 97 pgs: 97 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-05-09T08:34:52.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:52 smithi133 ceph-mon[29610]: pgmap v525: 97 pgs: 97 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-05-09T08:34:52.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 165384 7.96 MB/sec execute 68 sec latency 9.635 ms 2024-05-09T08:34:52.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:52 smithi188 ceph-mon[34645]: pgmap v525: 97 pgs: 97 active+clean; 78 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.1 MiB/s wr, 213 op/s 2024-05-09T08:34:53.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 166692 8.01 MB/sec execute 69 sec latency 25.839 ms 2024-05-09T08:34:54.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:54 smithi160 ceph-mon[34070]: pgmap v526: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-05-09T08:34:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:54 smithi133 ceph-mon[29610]: pgmap v526: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-05-09T08:34:54.927 INFO:tasks.workunit.client.0.smithi133.stdout: 1 168142 7.99 MB/sec execute 70 sec latency 11.821 ms 2024-05-09T08:34:54.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:54 smithi188 ceph-mon[34645]: pgmap v526: 97 pgs: 97 active+clean; 78 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-05-09T08:34:55.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 169329 7.92 MB/sec execute 71 sec latency 45.550 ms 2024-05-09T08:34:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:56 smithi160 ceph-mon[34070]: pgmap v527: 97 pgs: 97 active+clean; 81 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-05-09T08:34:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:56 smithi133 ceph-mon[29610]: pgmap v527: 97 pgs: 97 active+clean; 81 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-05-09T08:34:56.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 170939 7.97 MB/sec execute 72 sec latency 23.218 ms 2024-05-09T08:34:56.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:56 smithi188 ceph-mon[34645]: pgmap v527: 97 pgs: 97 active+clean; 81 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-05-09T08:34:57.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 172728 7.96 MB/sec execute 73 sec latency 19.977 ms 2024-05-09T08:34:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:34:58 smithi160 ceph-mon[34070]: pgmap v528: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-09T08:34:58.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:34:58 smithi133 ceph-mon[29610]: pgmap v528: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-09T08:34:58.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 174160 8.01 MB/sec execute 74 sec latency 23.352 ms 2024-05-09T08:34:58.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:34:58 smithi188 ceph-mon[34645]: pgmap v528: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-09T08:34:59.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 175712 7.98 MB/sec execute 75 sec latency 19.538 ms 2024-05-09T08:35:00.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:00 smithi160 ceph-mon[34070]: pgmap v529: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:00.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:00 smithi133 ceph-mon[29610]: pgmap v529: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:00.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 176838 7.97 MB/sec execute 76 sec latency 49.209 ms 2024-05-09T08:35:00.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:00 smithi188 ceph-mon[34645]: pgmap v529: 97 pgs: 97 active+clean; 81 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:01.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 178155 7.95 MB/sec execute 77 sec latency 20.404 ms 2024-05-09T08:35:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:02 smithi160 ceph-mon[34070]: pgmap v530: 97 pgs: 97 active+clean; 83 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-05-09T08:35:02.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:02 smithi133 ceph-mon[29610]: pgmap v530: 97 pgs: 97 active+clean; 83 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-05-09T08:35:02.928 INFO:tasks.workunit.client.0.smithi133.stdout: 1 179807 7.94 MB/sec execute 78 sec latency 22.045 ms 2024-05-09T08:35:02.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:02 smithi188 ceph-mon[34645]: pgmap v530: 97 pgs: 97 active+clean; 83 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-05-09T08:35:03.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 181197 7.98 MB/sec execute 79 sec latency 25.573 ms 2024-05-09T08:35:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:04 smithi160 ceph-mon[34070]: pgmap v531: 97 pgs: 97 active+clean; 83 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:04 smithi133 ceph-mon[29610]: pgmap v531: 97 pgs: 97 active+clean; 83 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:04.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 182688 7.95 MB/sec execute 80 sec latency 16.626 ms 2024-05-09T08:35:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:04 smithi188 ceph-mon[34645]: pgmap v531: 97 pgs: 97 active+clean; 83 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.2 MiB/s wr, 223 op/s 2024-05-09T08:35:05.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 183911 7.95 MB/sec execute 81 sec latency 46.399 ms 2024-05-09T08:35:06.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:06 smithi160 ceph-mon[34070]: pgmap v532: 97 pgs: 97 active+clean; 84 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:06 smithi133 ceph-mon[29610]: pgmap v532: 97 pgs: 97 active+clean; 84 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:06.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 185266 7.93 MB/sec execute 82 sec latency 17.518 ms 2024-05-09T08:35:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:06 smithi188 ceph-mon[34645]: pgmap v532: 97 pgs: 97 active+clean; 84 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:07.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 186886 7.92 MB/sec execute 83 sec latency 24.668 ms 2024-05-09T08:35:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:08 smithi160 ceph-mon[34070]: pgmap v533: 97 pgs: 97 active+clean; 86 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-09T08:35:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:08 smithi133 ceph-mon[29610]: pgmap v533: 97 pgs: 97 active+clean; 86 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-09T08:35:08.929 INFO:tasks.workunit.client.0.smithi133.stdout: 1 188225 7.96 MB/sec execute 84 sec latency 25.369 ms 2024-05-09T08:35:08.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:08 smithi188 ceph-mon[34645]: pgmap v533: 97 pgs: 97 active+clean; 86 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-05-09T08:35:09.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 189782 7.94 MB/sec execute 85 sec latency 12.853 ms 2024-05-09T08:35:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:10 smithi160 ceph-mon[34070]: pgmap v534: 97 pgs: 97 active+clean; 86 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:35:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:10 smithi133 ceph-mon[29610]: pgmap v534: 97 pgs: 97 active+clean; 86 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:35:10.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 191292 7.95 MB/sec execute 86 sec latency 54.338 ms 2024-05-09T08:35:10.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:10 smithi188 ceph-mon[34645]: pgmap v534: 97 pgs: 97 active+clean; 86 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:35:11.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 192701 7.94 MB/sec execute 87 sec latency 15.640 ms 2024-05-09T08:35:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:12 smithi160 ceph-mon[34070]: pgmap v535: 97 pgs: 97 active+clean; 86 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:35:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:12 smithi133 ceph-mon[29610]: pgmap v535: 97 pgs: 97 active+clean; 86 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:35:12.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 194143 7.91 MB/sec execute 88 sec latency 20.505 ms 2024-05-09T08:35:12.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:12 smithi188 ceph-mon[34645]: pgmap v535: 97 pgs: 97 active+clean; 86 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:35:13.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 195778 7.95 MB/sec execute 89 sec latency 20.608 ms 2024-05-09T08:35:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:14 smithi160 ceph-mon[34070]: pgmap v536: 97 pgs: 97 active+clean; 87 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-05-09T08:35:14.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:14 smithi133 ceph-mon[29610]: pgmap v536: 97 pgs: 97 active+clean; 87 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-05-09T08:35:14.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 197544 7.94 MB/sec execute 90 sec latency 20.788 ms 2024-05-09T08:35:14.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:14 smithi188 ceph-mon[34645]: pgmap v536: 97 pgs: 97 active+clean; 87 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-05-09T08:35:15.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 198961 7.98 MB/sec execute 91 sec latency 46.995 ms 2024-05-09T08:35:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:16 smithi160 ceph-mon[34070]: pgmap v537: 97 pgs: 97 active+clean; 90 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:16.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:16 smithi133 ceph-mon[29610]: pgmap v537: 97 pgs: 97 active+clean; 90 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:16.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 200134 7.95 MB/sec execute 92 sec latency 199.392 ms 2024-05-09T08:35:16.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:16 smithi188 ceph-mon[34645]: pgmap v537: 97 pgs: 97 active+clean; 90 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:17.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 201144 7.90 MB/sec execute 93 sec latency 260.681 ms 2024-05-09T08:35:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:18 smithi160 ceph-mon[34070]: pgmap v538: 97 pgs: 97 active+clean; 91 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:35:18.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:18 smithi133 ceph-mon[29610]: pgmap v538: 97 pgs: 97 active+clean; 91 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:35:18.930 INFO:tasks.workunit.client.0.smithi133.stdout: 1 202447 7.93 MB/sec execute 94 sec latency 558.669 ms 2024-05-09T08:35:18.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:18 smithi188 ceph-mon[34645]: pgmap v538: 97 pgs: 97 active+clean; 91 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-05-09T08:35:19.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 203962 7.91 MB/sec execute 95 sec latency 15.221 ms 2024-05-09T08:35:20.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:20 smithi133 ceph-mon[29610]: pgmap v539: 97 pgs: 97 active+clean; 91 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-05-09T08:35:20.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 205116 7.90 MB/sec execute 96 sec latency 47.927 ms 2024-05-09T08:35:20.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:20 smithi188 ceph-mon[34645]: pgmap v539: 97 pgs: 97 active+clean; 91 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-05-09T08:35:21.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:20 smithi160 ceph-mon[34070]: pgmap v539: 97 pgs: 97 active+clean; 91 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1.8 MiB/s wr, 169 op/s 2024-05-09T08:35:21.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 206556 7.89 MB/sec execute 97 sec latency 15.985 ms 2024-05-09T08:35:22.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:22 smithi133 ceph-mon[29610]: pgmap v540: 97 pgs: 97 active+clean; 93 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-05-09T08:35:22.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 208170 7.88 MB/sec execute 98 sec latency 25.762 ms 2024-05-09T08:35:22.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:22 smithi188 ceph-mon[34645]: pgmap v540: 97 pgs: 97 active+clean; 93 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-05-09T08:35:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:22 smithi160 ceph-mon[34070]: pgmap v540: 97 pgs: 97 active+clean; 93 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-05-09T08:35:23.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 209562 7.92 MB/sec execute 99 sec latency 27.810 ms 2024-05-09T08:35:24.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:24 smithi133 ceph-mon[29610]: pgmap v541: 97 pgs: 97 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-05-09T08:35:24.931 INFO:tasks.workunit.client.0.smithi133.stdout: 1 211043 7.89 MB/sec execute 100 sec latency 15.327 ms 2024-05-09T08:35:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:24 smithi188 ceph-mon[34645]: pgmap v541: 97 pgs: 97 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-05-09T08:35:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:24 smithi160 ceph-mon[34070]: pgmap v541: 97 pgs: 97 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-05-09T08:35:25.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 212177 7.88 MB/sec execute 101 sec latency 46.140 ms 2024-05-09T08:35:26.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:26 smithi133 ceph-mon[29610]: pgmap v542: 97 pgs: 97 active+clean; 95 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-09T08:35:26.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 213609 7.88 MB/sec execute 102 sec latency 19.393 ms 2024-05-09T08:35:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:26 smithi188 ceph-mon[34645]: pgmap v542: 97 pgs: 97 active+clean; 95 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-09T08:35:27.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:26 smithi160 ceph-mon[34070]: pgmap v542: 97 pgs: 97 active+clean; 95 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-09T08:35:27.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 215312 7.87 MB/sec execute 103 sec latency 25.703 ms 2024-05-09T08:35:28.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:28 smithi188 ceph-mon[34645]: pgmap v543: 97 pgs: 97 active+clean; 94 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:35:28.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 216698 7.90 MB/sec execute 104 sec latency 22.744 ms 2024-05-09T08:35:29.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:28 smithi160 ceph-mon[34070]: pgmap v543: 97 pgs: 97 active+clean; 94 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:35:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:28 smithi133 ceph-mon[29610]: pgmap v543: 97 pgs: 97 active+clean; 94 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:35:29.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 218230 7.88 MB/sec execute 105 sec latency 17.871 ms 2024-05-09T08:35:30.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 218852 7.82 MB/sec execute 106 sec latency 502.563 ms 2024-05-09T08:35:31.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:30 smithi160 ceph-mon[34070]: pgmap v544: 97 pgs: 97 active+clean; 96 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-05-09T08:35:31.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:30 smithi133 ceph-mon[29610]: pgmap v544: 97 pgs: 97 active+clean; 96 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-05-09T08:35:31.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:30 smithi188 ceph-mon[34645]: pgmap v544: 97 pgs: 97 active+clean; 96 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 211 op/s 2024-05-09T08:35:31.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 220200 7.86 MB/sec execute 107 sec latency 75.684 ms 2024-05-09T08:35:32.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 221488 7.84 MB/sec execute 108 sec latency 18.029 ms 2024-05-09T08:35:33.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:32 smithi160 ceph-mon[34070]: pgmap v545: 97 pgs: 97 active+clean; 96 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-09T08:35:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:32 smithi133 ceph-mon[29610]: pgmap v545: 97 pgs: 97 active+clean; 96 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-09T08:35:33.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:32 smithi188 ceph-mon[34645]: pgmap v545: 97 pgs: 97 active+clean; 96 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.0 MiB/s wr, 204 op/s 2024-05-09T08:35:33.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 222470 7.79 MB/sec execute 109 sec latency 226.046 ms 2024-05-09T08:35:34.932 INFO:tasks.workunit.client.0.smithi133.stdout: 1 224047 7.82 MB/sec execute 110 sec latency 234.185 ms 2024-05-09T08:35:35.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:34 smithi160 ceph-mon[34070]: pgmap v546: 97 pgs: 97 active+clean; 97 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-09T08:35:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:34 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:35:35.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:34 smithi133 ceph-mon[29610]: pgmap v546: 97 pgs: 97 active+clean; 97 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-09T08:35:35.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:34 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:35:35.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:34 smithi188 ceph-mon[34645]: pgmap v546: 97 pgs: 97 active+clean; 97 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.0 MiB/s wr, 215 op/s 2024-05-09T08:35:35.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:34 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:35:35.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 225904 7.82 MB/sec execute 111 sec latency 23.105 ms 2024-05-09T08:35:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:35:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:35:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:35:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:35:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:35:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:35:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:35:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:35:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:35:36.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 227189 7.84 MB/sec execute 112 sec latency 61.414 ms 2024-05-09T08:35:37.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:36 smithi160 ceph-mon[34070]: pgmap v547: 97 pgs: 97 active+clean; 99 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:37.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:36 smithi133 ceph-mon[29610]: pgmap v547: 97 pgs: 97 active+clean; 99 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:37.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:36 smithi188 ceph-mon[34645]: pgmap v547: 97 pgs: 97 active+clean; 99 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2024-05-09T08:35:37.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 228318 7.80 MB/sec execute 113 sec latency 15.806 ms 2024-05-09T08:35:38.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 229626 7.79 MB/sec execute 114 sec latency 26.696 ms 2024-05-09T08:35:39.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:38 smithi160 ceph-mon[34070]: pgmap v548: 97 pgs: 97 active+clean; 99 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-05-09T08:35:39.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:38 smithi133 ceph-mon[29610]: pgmap v548: 97 pgs: 97 active+clean; 99 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-05-09T08:35:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:38 smithi188 ceph-mon[34645]: pgmap v548: 97 pgs: 97 active+clean; 99 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-05-09T08:35:39.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 231058 7.81 MB/sec execute 115 sec latency 24.134 ms 2024-05-09T08:35:40.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 232914 7.81 MB/sec execute 116 sec latency 16.515 ms 2024-05-09T08:35:40.983 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:40 smithi188 ceph-mon[34645]: pgmap v549: 97 pgs: 97 active+clean; 99 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:35:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:40 smithi160 ceph-mon[34070]: pgmap v549: 97 pgs: 97 active+clean; 99 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:35:41.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:40 smithi133 ceph-mon[29610]: pgmap v549: 97 pgs: 97 active+clean; 99 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:35:41.933 INFO:tasks.workunit.client.0.smithi133.stdout: 1 233907 7.80 MB/sec execute 117 sec latency 43.179 ms 2024-05-09T08:35:42.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 235071 7.79 MB/sec execute 118 sec latency 13.239 ms 2024-05-09T08:35:43.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:42 smithi160 ceph-mon[34070]: pgmap v550: 97 pgs: 97 active+clean; 101 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:35:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:42 smithi133 ceph-mon[29610]: pgmap v550: 97 pgs: 97 active+clean; 101 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:35:43.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:42 smithi188 ceph-mon[34645]: pgmap v550: 97 pgs: 97 active+clean; 101 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-05-09T08:35:43.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 236616 7.78 MB/sec execute 119 sec latency 26.515 ms 2024-05-09T08:35:44.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 238039 7.81 MB/sec execute 120 sec latency 26.846 ms 2024-05-09T08:35:45.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:44 smithi160 ceph-mon[34070]: pgmap v551: 97 pgs: 97 active+clean; 102 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:45.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:44 smithi133 ceph-mon[29610]: pgmap v551: 97 pgs: 97 active+clean; 102 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:45.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:44 smithi188 ceph-mon[34645]: pgmap v551: 97 pgs: 97 active+clean; 102 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:45.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 239612 7.79 MB/sec execute 121 sec latency 18.479 ms 2024-05-09T08:35:46.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 240712 7.78 MB/sec execute 122 sec latency 55.007 ms 2024-05-09T08:35:47.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:46 smithi160 ceph-mon[34070]: pgmap v552: 97 pgs: 97 active+clean; 105 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-09T08:35:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:46 smithi133 ceph-mon[29610]: pgmap v552: 97 pgs: 97 active+clean; 105 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-09T08:35:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:46 smithi188 ceph-mon[34645]: pgmap v552: 97 pgs: 97 active+clean; 105 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 217 op/s 2024-05-09T08:35:47.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 242023 7.78 MB/sec execute 123 sec latency 18.071 ms 2024-05-09T08:35:48.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 243688 7.77 MB/sec execute 124 sec latency 21.960 ms 2024-05-09T08:35:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:48 smithi160 ceph-mon[34070]: pgmap v553: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:35:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:48 smithi133 ceph-mon[29610]: pgmap v553: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:35:49.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:48 smithi188 ceph-mon[34645]: pgmap v553: 97 pgs: 97 active+clean; 104 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:35:49.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 245007 7.80 MB/sec execute 125 sec latency 28.800 ms 2024-05-09T08:35:50.934 INFO:tasks.workunit.client.0.smithi133.stdout: 1 246426 7.78 MB/sec execute 126 sec latency 14.433 ms 2024-05-09T08:35:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:50 smithi160 ceph-mon[34070]: pgmap v554: 97 pgs: 97 active+clean; 104 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-09T08:35:51.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:50 smithi133 ceph-mon[29610]: pgmap v554: 97 pgs: 97 active+clean; 104 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-09T08:35:51.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:50 smithi188 ceph-mon[34645]: pgmap v554: 97 pgs: 97 active+clean; 104 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-09T08:35:51.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 247357 7.74 MB/sec execute 127 sec latency 218.833 ms 2024-05-09T08:35:52.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 248935 7.77 MB/sec execute 128 sec latency 20.659 ms 2024-05-09T08:35:53.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:52 smithi160 ceph-mon[34070]: pgmap v555: 97 pgs: 97 active+clean; 106 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:53.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:52 smithi133 ceph-mon[29610]: pgmap v555: 97 pgs: 97 active+clean; 106 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:53.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:52 smithi188 ceph-mon[34645]: pgmap v555: 97 pgs: 97 active+clean; 106 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-05-09T08:35:53.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 250758 7.76 MB/sec execute 129 sec latency 20.457 ms 2024-05-09T08:35:54.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 252258 7.80 MB/sec execute 130 sec latency 20.452 ms 2024-05-09T08:35:55.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:54 smithi160 ceph-mon[34070]: pgmap v556: 97 pgs: 97 active+clean; 106 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:35:55.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:54 smithi133 ceph-mon[29610]: pgmap v556: 97 pgs: 97 active+clean; 106 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:35:55.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:54 smithi188 ceph-mon[34645]: pgmap v556: 97 pgs: 97 active+clean; 106 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:35:55.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 254215 7.79 MB/sec execute 131 sec latency 13.659 ms 2024-05-09T08:35:56.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 255427 7.81 MB/sec execute 132 sec latency 39.556 ms 2024-05-09T08:35:57.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:56 smithi160 ceph-mon[34070]: pgmap v557: 97 pgs: 97 active+clean; 108 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-05-09T08:35:57.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:56 smithi133 ceph-mon[29610]: pgmap v557: 97 pgs: 97 active+clean; 108 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-05-09T08:35:57.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:56 smithi188 ceph-mon[34645]: pgmap v557: 97 pgs: 97 active+clean; 108 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-05-09T08:35:57.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 256679 7.78 MB/sec execute 133 sec latency 11.440 ms 2024-05-09T08:35:58.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 258041 7.77 MB/sec execute 134 sec latency 22.966 ms 2024-05-09T08:35:59.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:35:58 smithi160 ceph-mon[34070]: pgmap v558: 97 pgs: 97 active+clean; 107 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:35:59.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:35:58 smithi133 ceph-mon[29610]: pgmap v558: 97 pgs: 97 active+clean; 107 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:35:59.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:35:58 smithi188 ceph-mon[34645]: pgmap v558: 97 pgs: 97 active+clean; 107 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:35:59.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 259702 7.79 MB/sec execute 135 sec latency 15.523 ms 2024-05-09T08:36:00.935 INFO:tasks.workunit.client.0.smithi133.stdout: 1 261376 7.78 MB/sec execute 136 sec latency 20.929 ms 2024-05-09T08:36:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:00 smithi160 ceph-mon[34070]: pgmap v559: 97 pgs: 97 active+clean; 107 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:36:01.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:00 smithi133 ceph-mon[29610]: pgmap v559: 97 pgs: 97 active+clean; 107 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:36:01.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:00 smithi188 ceph-mon[34645]: pgmap v559: 97 pgs: 97 active+clean; 107 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 183 op/s 2024-05-09T08:36:01.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 262608 7.81 MB/sec execute 137 sec latency 28.860 ms 2024-05-09T08:36:02.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 263700 7.77 MB/sec execute 138 sec latency 14.909 ms 2024-05-09T08:36:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:02 smithi160 ceph-mon[34070]: pgmap v560: 97 pgs: 97 active+clean; 109 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-05-09T08:36:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:02 smithi133 ceph-mon[29610]: pgmap v560: 97 pgs: 97 active+clean; 109 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-05-09T08:36:03.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:02 smithi188 ceph-mon[34645]: pgmap v560: 97 pgs: 97 active+clean; 109 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-05-09T08:36:03.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 265011 7.76 MB/sec execute 139 sec latency 26.343 ms 2024-05-09T08:36:04.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 266384 7.78 MB/sec execute 140 sec latency 23.054 ms 2024-05-09T08:36:05.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:04 smithi160 ceph-mon[34070]: pgmap v561: 97 pgs: 97 active+clean; 111 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-09T08:36:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:04 smithi133 ceph-mon[29610]: pgmap v561: 97 pgs: 97 active+clean; 111 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-09T08:36:05.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:04 smithi188 ceph-mon[34645]: pgmap v561: 97 pgs: 97 active+clean; 111 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-09T08:36:05.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 267622 7.77 MB/sec execute 141 sec latency 17.280 ms 2024-05-09T08:36:06.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 268664 7.73 MB/sec execute 142 sec latency 47.434 ms 2024-05-09T08:36:07.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:06 smithi160 ceph-mon[34070]: pgmap v562: 97 pgs: 97 active+clean; 113 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-05-09T08:36:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:06 smithi133 ceph-mon[29610]: pgmap v562: 97 pgs: 97 active+clean; 113 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-05-09T08:36:07.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:06 smithi188 ceph-mon[34645]: pgmap v562: 97 pgs: 97 active+clean; 113 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-05-09T08:36:07.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 270074 7.76 MB/sec execute 143 sec latency 28.458 ms 2024-05-09T08:36:08.936 INFO:tasks.workunit.client.0.smithi133.stdout: 1 271807 7.74 MB/sec execute 144 sec latency 13.767 ms 2024-05-09T08:36:09.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:08 smithi160 ceph-mon[34070]: pgmap v563: 97 pgs: 97 active+clean; 113 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:36:09.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:08 smithi133 ceph-mon[29610]: pgmap v563: 97 pgs: 97 active+clean; 113 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:36:09.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:08 smithi188 ceph-mon[34645]: pgmap v563: 97 pgs: 97 active+clean; 113 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-05-09T08:36:09.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 272809 7.74 MB/sec execute 145 sec latency 24.533 ms 2024-05-09T08:36:10.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 273978 7.73 MB/sec execute 146 sec latency 29.721 ms 2024-05-09T08:36:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:10 smithi160 ceph-mon[34070]: pgmap v564: 97 pgs: 97 active+clean; 113 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-09T08:36:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:10 smithi133 ceph-mon[29610]: pgmap v564: 97 pgs: 97 active+clean; 113 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-09T08:36:11.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:10 smithi188 ceph-mon[34645]: pgmap v564: 97 pgs: 97 active+clean; 113 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-09T08:36:11.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 275642 7.73 MB/sec execute 147 sec latency 56.333 ms 2024-05-09T08:36:12.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 277242 7.75 MB/sec execute 148 sec latency 25.856 ms 2024-05-09T08:36:13.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:12 smithi160 ceph-mon[34070]: pgmap v565: 97 pgs: 97 active+clean; 115 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:36:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:12 smithi133 ceph-mon[29610]: pgmap v565: 97 pgs: 97 active+clean; 115 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:36:13.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:12 smithi188 ceph-mon[34645]: pgmap v565: 97 pgs: 97 active+clean; 115 MiB data, 953 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:36:13.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 279021 7.75 MB/sec execute 149 sec latency 188.751 ms 2024-05-09T08:36:14.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 280357 7.77 MB/sec execute 150 sec latency 20.359 ms 2024-05-09T08:36:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:14 smithi160 ceph-mon[34070]: pgmap v566: 97 pgs: 97 active+clean; 115 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:36:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:14 smithi133 ceph-mon[29610]: pgmap v566: 97 pgs: 97 active+clean; 115 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:36:15.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:14 smithi188 ceph-mon[34645]: pgmap v566: 97 pgs: 97 active+clean; 115 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:36:15.937 INFO:tasks.workunit.client.0.smithi133.stdout: 1 281880 7.76 MB/sec execute 151 sec latency 14.616 ms 2024-05-09T08:36:16.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 283246 7.76 MB/sec execute 152 sec latency 24.095 ms 2024-05-09T08:36:17.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:16 smithi160 ceph-mon[34070]: pgmap v567: 97 pgs: 97 active+clean; 116 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-05-09T08:36:17.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:16 smithi133 ceph-mon[29610]: pgmap v567: 97 pgs: 97 active+clean; 116 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-05-09T08:36:17.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:16 smithi188 ceph-mon[34645]: pgmap v567: 97 pgs: 97 active+clean; 116 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-05-09T08:36:17.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 284753 7.75 MB/sec execute 153 sec latency 30.325 ms 2024-05-09T08:36:18.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 286351 7.75 MB/sec execute 154 sec latency 20.510 ms 2024-05-09T08:36:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:18 smithi160 ceph-mon[34070]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:36:19.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:18 smithi133 ceph-mon[29610]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:36:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:18 smithi188 ceph-mon[34645]: pgmap v568: 97 pgs: 97 active+clean; 118 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:36:19.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 288026 7.77 MB/sec execute 155 sec latency 20.630 ms 2024-05-09T08:36:20.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 289636 7.77 MB/sec execute 156 sec latency 324.000 ms 2024-05-09T08:36:21.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:20 smithi160 ceph-mon[34070]: pgmap v569: 97 pgs: 97 active+clean; 119 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:36:21.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:20 smithi133 ceph-mon[29610]: pgmap v569: 97 pgs: 97 active+clean; 119 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:36:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:20 smithi188 ceph-mon[34645]: pgmap v569: 97 pgs: 97 active+clean; 119 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:36:21.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 290605 7.76 MB/sec execute 157 sec latency 440.836 ms 2024-05-09T08:36:22.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 291865 7.75 MB/sec execute 158 sec latency 35.755 ms 2024-05-09T08:36:23.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:22 smithi160 ceph-mon[34070]: pgmap v570: 97 pgs: 97 active+clean; 119 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:36:23.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:22 smithi133 ceph-mon[29610]: pgmap v570: 97 pgs: 97 active+clean; 119 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:36:23.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:22 smithi188 ceph-mon[34645]: pgmap v570: 97 pgs: 97 active+clean; 119 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 199 op/s 2024-05-09T08:36:23.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 293428 7.75 MB/sec execute 159 sec latency 20.584 ms 2024-05-09T08:36:24.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 294829 7.77 MB/sec execute 160 sec latency 69.343 ms 2024-05-09T08:36:25.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:24 smithi160 ceph-mon[34070]: pgmap v571: 97 pgs: 97 active+clean; 120 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-09T08:36:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:24 smithi133 ceph-mon[29610]: pgmap v571: 97 pgs: 97 active+clean; 120 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-09T08:36:25.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:24 smithi188 ceph-mon[34645]: pgmap v571: 97 pgs: 97 active+clean; 120 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-05-09T08:36:25.938 INFO:tasks.workunit.client.0.smithi133.stdout: 1 296779 7.76 MB/sec execute 161 sec latency 14.036 ms 2024-05-09T08:36:26.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 297355 7.75 MB/sec execute 162 sec latency 326.593 ms 2024-05-09T08:36:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:26 smithi160 ceph-mon[34070]: pgmap v572: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-05-09T08:36:27.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:26 smithi133 ceph-mon[29610]: pgmap v572: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-05-09T08:36:27.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:26 smithi188 ceph-mon[34645]: pgmap v572: 97 pgs: 97 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.0 MiB/s wr, 217 op/s 2024-05-09T08:36:27.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 298730 7.74 MB/sec execute 163 sec latency 57.914 ms 2024-05-09T08:36:28.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 300453 7.74 MB/sec execute 164 sec latency 19.385 ms 2024-05-09T08:36:29.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:28 smithi160 ceph-mon[34070]: pgmap v573: 97 pgs: 97 active+clean; 123 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-05-09T08:36:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:28 smithi133 ceph-mon[29610]: pgmap v573: 97 pgs: 97 active+clean; 123 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-05-09T08:36:29.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:28 smithi188 ceph-mon[34645]: pgmap v573: 97 pgs: 97 active+clean; 123 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-05-09T08:36:29.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 301898 7.76 MB/sec execute 165 sec latency 133.044 ms 2024-05-09T08:36:30.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 303832 7.76 MB/sec execute 166 sec latency 18.167 ms 2024-05-09T08:36:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:30 smithi160 ceph-mon[34070]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T08:36:31.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:30 smithi133 ceph-mon[29610]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T08:36:31.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:30 smithi188 ceph-mon[34645]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-05-09T08:36:31.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 305097 7.78 MB/sec execute 167 sec latency 24.048 ms 2024-05-09T08:36:32.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 306517 7.77 MB/sec execute 168 sec latency 28.785 ms 2024-05-09T08:36:33.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:32 smithi160 ceph-mon[34070]: pgmap v575: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-05-09T08:36:33.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:32 smithi133 ceph-mon[29610]: pgmap v575: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-05-09T08:36:33.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:32 smithi188 ceph-mon[34645]: pgmap v575: 97 pgs: 97 active+clean; 122 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-05-09T08:36:33.939 INFO:tasks.workunit.client.0.smithi133.stdout: 1 307631 7.74 MB/sec execute 169 sec latency 19.890 ms 2024-05-09T08:36:34.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 309345 7.76 MB/sec execute 170 sec latency 19.811 ms 2024-05-09T08:36:35.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:34 smithi160 ceph-mon[34070]: pgmap v576: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-09T08:36:35.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:34 smithi133 ceph-mon[29610]: pgmap v576: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-09T08:36:35.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:34 smithi188 ceph-mon[34645]: pgmap v576: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-05-09T08:36:35.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 310982 7.76 MB/sec execute 171 sec latency 288.773 ms 2024-05-09T08:36:36.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 312359 7.78 MB/sec execute 172 sec latency 224.909 ms 2024-05-09T08:36:37.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: pgmap v577: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:36:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:36:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:36 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: pgmap v577: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:36 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: pgmap v577: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:36:37.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:36:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:36 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:37.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 314045 7.77 MB/sec execute 173 sec latency 38.767 ms 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: pgmap v578: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:36:38.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:36:38.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:36:38.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:37 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:36:38.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: pgmap v578: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:36:38.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:37 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:36:38.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:36:38.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:36:38.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: pgmap v578: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:36:38.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:37 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:36:38.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 315146 7.76 MB/sec execute 174 sec latency 77.249 ms 2024-05-09T08:36:39.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 316648 7.76 MB/sec execute 175 sec latency 14.174 ms 2024-05-09T08:36:40.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:39 smithi160 ceph-mon[34070]: pgmap v579: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-09T08:36:40.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:39 smithi133 ceph-mon[29610]: pgmap v579: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-09T08:36:40.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:39 smithi188 ceph-mon[34645]: pgmap v579: 97 pgs: 97 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-05-09T08:36:40.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 318172 7.76 MB/sec execute 176 sec latency 26.591 ms 2024-05-09T08:36:41.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 319622 7.78 MB/sec execute 177 sec latency 22.434 ms 2024-05-09T08:36:42.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:41 smithi188 ceph-mon[34645]: pgmap v580: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-05-09T08:36:42.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:41 smithi160 ceph-mon[34070]: pgmap v580: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-05-09T08:36:42.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:41 smithi133 ceph-mon[29610]: pgmap v580: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-05-09T08:36:42.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 321112 7.76 MB/sec execute 178 sec latency 54.879 ms 2024-05-09T08:36:43.940 INFO:tasks.workunit.client.0.smithi133.stdout: 1 322244 7.76 MB/sec execute 179 sec latency 24.827 ms 2024-05-09T08:36:44.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:43 smithi160 ceph-mon[34070]: pgmap v581: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:44.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:43 smithi133 ceph-mon[29610]: pgmap v581: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:44.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:43 smithi188 ceph-mon[34645]: pgmap v581: 97 pgs: 97 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-05-09T08:36:44.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 323660 7.76 MB/sec execute 180 sec latency 16.989 ms 2024-05-09T08:36:45.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 325245 7.75 MB/sec execute 181 sec latency 21.556 ms 2024-05-09T08:36:46.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:45 smithi160 ceph-mon[34070]: pgmap v582: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-09T08:36:46.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:45 smithi133 ceph-mon[29610]: pgmap v582: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-09T08:36:46.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:45 smithi188 ceph-mon[34645]: pgmap v582: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-09T08:36:46.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 326700 7.77 MB/sec execute 182 sec latency 23.041 ms 2024-05-09T08:36:47.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 327946 7.76 MB/sec execute 183 sec latency 63.762 ms 2024-05-09T08:36:48.232 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:47 smithi133 ceph-mon[29610]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:48.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:47 smithi160 ceph-mon[34070]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:48.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:47 smithi188 ceph-mon[34645]: pgmap v583: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:48.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 329120 7.75 MB/sec execute 184 sec latency 25.338 ms 2024-05-09T08:36:49.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 330480 7.75 MB/sec execute 185 sec latency 19.673 ms 2024-05-09T08:36:50.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:49 smithi160 ceph-mon[34070]: pgmap v584: 97 pgs: 97 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:36:50.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:49 smithi133 ceph-mon[29610]: pgmap v584: 97 pgs: 97 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:36:50.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:49 smithi188 ceph-mon[34645]: pgmap v584: 97 pgs: 97 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-05-09T08:36:50.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 332254 7.75 MB/sec execute 186 sec latency 21.797 ms 2024-05-09T08:36:51.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 333395 7.76 MB/sec execute 187 sec latency 29.712 ms 2024-05-09T08:36:52.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:51 smithi160 ceph-mon[34070]: pgmap v585: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-09T08:36:52.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:51 smithi133 ceph-mon[29610]: pgmap v585: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-09T08:36:52.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:51 smithi188 ceph-mon[34645]: pgmap v585: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-05-09T08:36:52.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 334418 7.73 MB/sec execute 188 sec latency 76.815 ms 2024-05-09T08:36:53.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 335935 7.73 MB/sec execute 189 sec latency 44.608 ms 2024-05-09T08:36:54.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:53 smithi160 ceph-mon[34070]: pgmap v586: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:54.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:53 smithi133 ceph-mon[29610]: pgmap v586: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:54.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:53 smithi188 ceph-mon[34645]: pgmap v586: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:36:54.941 INFO:tasks.workunit.client.0.smithi133.stdout: 1 337335 7.75 MB/sec execute 190 sec latency 75.824 ms 2024-05-09T08:36:55.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 338753 7.74 MB/sec execute 191 sec latency 15.823 ms 2024-05-09T08:36:56.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:56 smithi160 ceph-mon[34070]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:36:56.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:56 smithi133 ceph-mon[29610]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:36:56.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:56 smithi188 ceph-mon[34645]: pgmap v587: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:36:56.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 339949 7.73 MB/sec execute 192 sec latency 24.903 ms 2024-05-09T08:36:57.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 341241 7.73 MB/sec execute 193 sec latency 18.993 ms 2024-05-09T08:36:58.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:36:58 smithi160 ceph-mon[34070]: pgmap v588: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-05-09T08:36:58.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:36:58 smithi133 ceph-mon[29610]: pgmap v588: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-05-09T08:36:58.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:36:58 smithi188 ceph-mon[34645]: pgmap v588: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.0 MiB/s wr, 188 op/s 2024-05-09T08:36:58.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 342926 7.72 MB/sec execute 194 sec latency 50.797 ms 2024-05-09T08:36:59.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 344276 7.74 MB/sec execute 195 sec latency 24.105 ms 2024-05-09T08:37:00.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:00 smithi160 ceph-mon[34070]: pgmap v589: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:37:00.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:00 smithi133 ceph-mon[29610]: pgmap v589: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:37:00.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:00 smithi188 ceph-mon[34645]: pgmap v589: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 1.9 MiB/s wr, 187 op/s 2024-05-09T08:37:00.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 345762 7.73 MB/sec execute 196 sec latency 16.837 ms 2024-05-09T08:37:01.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 347253 7.74 MB/sec execute 197 sec latency 21.818 ms 2024-05-09T08:37:02.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:02 smithi160 ceph-mon[34070]: pgmap v590: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:37:02.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:02 smithi133 ceph-mon[29610]: pgmap v590: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:37:02.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:02 smithi188 ceph-mon[34645]: pgmap v590: 97 pgs: 97 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:37:02.942 INFO:tasks.workunit.client.0.smithi133.stdout: 1 348798 7.74 MB/sec execute 198 sec latency 10.353 ms 2024-05-09T08:37:03.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 350115 7.73 MB/sec execute 199 sec latency 72.998 ms 2024-05-09T08:37:04.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:04 smithi160 ceph-mon[34070]: pgmap v591: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-05-09T08:37:04.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:04 smithi133 ceph-mon[29610]: pgmap v591: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-05-09T08:37:04.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:04 smithi188 ceph-mon[34645]: pgmap v591: 97 pgs: 97 active+clean; 139 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-05-09T08:37:04.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 351458 7.74 MB/sec execute 200 sec latency 335.071 ms 2024-05-09T08:37:05.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 353450 7.74 MB/sec execute 201 sec latency 15.114 ms 2024-05-09T08:37:06.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:06 smithi160 ceph-mon[34070]: pgmap v592: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-05-09T08:37:06.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:06 smithi133 ceph-mon[29610]: pgmap v592: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-05-09T08:37:06.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:06 smithi188 ceph-mon[34645]: pgmap v592: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.3 MiB/s wr, 204 op/s 2024-05-09T08:37:06.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 354762 7.75 MB/sec execute 202 sec latency 22.584 ms 2024-05-09T08:37:07.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 356338 7.75 MB/sec execute 203 sec latency 9.831 ms 2024-05-09T08:37:08.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:08 smithi133 ceph-mon[29610]: pgmap v593: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-09T08:37:08.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:08 smithi188 ceph-mon[34645]: pgmap v593: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-09T08:37:08.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:08 smithi160 ceph-mon[34070]: pgmap v593: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-05-09T08:37:08.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 357698 7.75 MB/sec execute 204 sec latency 40.331 ms 2024-05-09T08:37:09.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 359223 7.75 MB/sec execute 205 sec latency 18.609 ms 2024-05-09T08:37:10.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:10 smithi188 ceph-mon[34645]: pgmap v594: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-09T08:37:10.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:10 smithi160 ceph-mon[34070]: pgmap v594: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-09T08:37:10.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:10 smithi133 ceph-mon[29610]: pgmap v594: 97 pgs: 97 active+clean; 142 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-05-09T08:37:10.943 INFO:tasks.workunit.client.0.smithi133.stdout: 1 360815 7.74 MB/sec execute 206 sec latency 23.154 ms 2024-05-09T08:37:11.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 362470 7.76 MB/sec execute 207 sec latency 21.119 ms 2024-05-09T08:37:12.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:12 smithi160 ceph-mon[34070]: pgmap v595: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-05-09T08:37:12.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:12 smithi133 ceph-mon[29610]: pgmap v595: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-05-09T08:37:12.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:12 smithi188 ceph-mon[34645]: pgmap v595: 97 pgs: 97 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-05-09T08:37:12.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 364240 7.76 MB/sec execute 208 sec latency 20.905 ms 2024-05-09T08:37:13.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 365740 7.78 MB/sec execute 209 sec latency 24.816 ms 2024-05-09T08:37:14.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:14 smithi160 ceph-mon[34070]: pgmap v596: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-05-09T08:37:14.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:14 smithi133 ceph-mon[29610]: pgmap v596: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-05-09T08:37:14.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:14 smithi188 ceph-mon[34645]: pgmap v596: 97 pgs: 97 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-05-09T08:37:14.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 367675 7.77 MB/sec execute 210 sec latency 15.716 ms 2024-05-09T08:37:15.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 369106 7.79 MB/sec execute 211 sec latency 19.313 ms 2024-05-09T08:37:16.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:16 smithi160 ceph-mon[34070]: pgmap v597: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-09T08:37:16.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:16 smithi133 ceph-mon[29610]: pgmap v597: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-09T08:37:16.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:16 smithi188 ceph-mon[34645]: pgmap v597: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 214 op/s 2024-05-09T08:37:16.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 370863 7.78 MB/sec execute 212 sec latency 13.195 ms 2024-05-09T08:37:17.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 372393 7.80 MB/sec execute 213 sec latency 20.624 ms 2024-05-09T08:37:18.463 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:18 smithi160 ceph-mon[34070]: pgmap v598: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-05-09T08:37:18.478 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:18 smithi133 ceph-mon[29610]: pgmap v598: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-05-09T08:37:18.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:18 smithi188 ceph-mon[34645]: pgmap v598: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-05-09T08:37:18.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 373619 7.78 MB/sec execute 214 sec latency 58.985 ms 2024-05-09T08:37:19.944 INFO:tasks.workunit.client.0.smithi133.stdout: 1 374936 7.77 MB/sec execute 215 sec latency 84.008 ms 2024-05-09T08:37:20.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:20 smithi160 ceph-mon[34070]: pgmap v599: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-05-09T08:37:20.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:20 smithi133 ceph-mon[29610]: pgmap v599: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-05-09T08:37:20.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:20 smithi188 ceph-mon[34645]: pgmap v599: 97 pgs: 97 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-05-09T08:37:20.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 376274 7.79 MB/sec execute 216 sec latency 190.612 ms 2024-05-09T08:37:21.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 377596 7.78 MB/sec execute 217 sec latency 17.822 ms 2024-05-09T08:37:22.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:22 smithi160 ceph-mon[34070]: pgmap v600: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-05-09T08:37:22.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:22 smithi133 ceph-mon[29610]: pgmap v600: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-05-09T08:37:22.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:22 smithi188 ceph-mon[34645]: pgmap v600: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-05-09T08:37:22.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 378773 7.77 MB/sec execute 218 sec latency 39.341 ms 2024-05-09T08:37:23.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 380102 7.77 MB/sec execute 219 sec latency 40.447 ms 2024-05-09T08:37:24.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:24 smithi160 ceph-mon[34070]: pgmap v601: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:37:24.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:24 smithi133 ceph-mon[29610]: pgmap v601: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:37:24.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:24 smithi188 ceph-mon[34645]: pgmap v601: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-05-09T08:37:24.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 381878 7.77 MB/sec execute 220 sec latency 19.101 ms 2024-05-09T08:37:25.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 382918 7.77 MB/sec execute 221 sec latency 23.191 ms 2024-05-09T08:37:26.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:26 smithi160 ceph-mon[34070]: pgmap v602: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:37:26.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:26 smithi133 ceph-mon[29610]: pgmap v602: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:37:26.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:26 smithi188 ceph-mon[34645]: pgmap v602: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-05-09T08:37:26.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 384031 7.76 MB/sec execute 222 sec latency 17.776 ms 2024-05-09T08:37:27.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 385597 7.75 MB/sec execute 223 sec latency 24.036 ms 2024-05-09T08:37:28.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:28 smithi160 ceph-mon[34070]: pgmap v603: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:37:28.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:28 smithi133 ceph-mon[29610]: pgmap v603: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:37:28.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:28 smithi188 ceph-mon[34645]: pgmap v603: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 205 op/s 2024-05-09T08:37:28.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 386957 7.77 MB/sec execute 224 sec latency 43.686 ms 2024-05-09T08:37:29.945 INFO:tasks.workunit.client.0.smithi133.stdout: 1 388371 7.76 MB/sec execute 225 sec latency 44.529 ms 2024-05-09T08:37:30.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:30 smithi160 ceph-mon[34070]: pgmap v604: 97 pgs: 97 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-05-09T08:37:30.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:30 smithi133 ceph-mon[29610]: pgmap v604: 97 pgs: 97 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-05-09T08:37:30.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:30 smithi188 ceph-mon[34645]: pgmap v604: 97 pgs: 97 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-05-09T08:37:30.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 389279 7.73 MB/sec execute 226 sec latency 117.464 ms 2024-05-09T08:37:31.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 390906 7.75 MB/sec execute 227 sec latency 14.253 ms 2024-05-09T08:37:32.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:32 smithi160 ceph-mon[34070]: pgmap v605: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-09T08:37:32.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:32 smithi133 ceph-mon[29610]: pgmap v605: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-09T08:37:32.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:32 smithi188 ceph-mon[34645]: pgmap v605: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-09T08:37:32.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 392688 7.75 MB/sec execute 228 sec latency 19.442 ms 2024-05-09T08:37:33.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 394085 7.76 MB/sec execute 229 sec latency 51.000 ms 2024-05-09T08:37:34.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:34 smithi160 ceph-mon[34070]: pgmap v606: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:34.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:34 smithi133 ceph-mon[29610]: pgmap v606: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:34.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:34 smithi188 ceph-mon[34645]: pgmap v606: 97 pgs: 97 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:34.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 396070 7.76 MB/sec execute 230 sec latency 15.790 ms 2024-05-09T08:37:35.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 397394 7.77 MB/sec execute 231 sec latency 22.722 ms 2024-05-09T08:37:36.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:36 smithi160 ceph-mon[34070]: pgmap v607: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:37:36.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:36 smithi133 ceph-mon[29610]: pgmap v607: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:37:36.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:36 smithi188 ceph-mon[34645]: pgmap v607: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 204 op/s 2024-05-09T08:37:36.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 398968 7.77 MB/sec execute 232 sec latency 14.601 ms 2024-05-09T08:37:37.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 400660 7.78 MB/sec execute 233 sec latency 32.056 ms 2024-05-09T08:37:38.661 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: pgmap v608: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:37:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:37:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:38 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: pgmap v608: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:37:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:37:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:38 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: pgmap v608: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-05-09T08:37:38.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:37:38.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:38 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:37:38.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 402079 7.77 MB/sec execute 234 sec latency 26.642 ms 2024-05-09T08:37:39.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:39 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:37:39.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:39 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:37:39.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:37:39.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:37:39.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:39 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:37:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:39 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:37:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:39 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:37:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:37:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:37:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:39 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:37:39.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:39 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:37:39.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:39 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:37:39.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:37:39.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:37:39.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:39 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:37:39.946 INFO:tasks.workunit.client.0.smithi133.stdout: 1 403460 7.76 MB/sec execute 235 sec latency 25.563 ms 2024-05-09T08:37:40.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 404961 7.78 MB/sec execute 236 sec latency 18.078 ms 2024-05-09T08:37:41.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:40 smithi160 ceph-mon[34070]: pgmap v609: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-05-09T08:37:41.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:40 smithi133 ceph-mon[29610]: pgmap v609: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-05-09T08:37:41.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:40 smithi188 ceph-mon[34645]: pgmap v609: 97 pgs: 97 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-05-09T08:37:41.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 406736 7.77 MB/sec execute 237 sec latency 13.183 ms 2024-05-09T08:37:42.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 407928 7.79 MB/sec execute 238 sec latency 21.646 ms 2024-05-09T08:37:43.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:42 smithi160 ceph-mon[34070]: pgmap v610: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-09T08:37:43.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:42 smithi133 ceph-mon[29610]: pgmap v610: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-09T08:37:43.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:42 smithi188 ceph-mon[34645]: pgmap v610: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-09T08:37:43.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 409072 7.77 MB/sec execute 239 sec latency 19.766 ms 2024-05-09T08:37:44.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 410378 7.76 MB/sec execute 240 sec latency 135.787 ms 2024-05-09T08:37:45.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:44 smithi160 ceph-mon[34070]: pgmap v611: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-09T08:37:45.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:44 smithi133 ceph-mon[29610]: pgmap v611: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-09T08:37:45.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:44 smithi188 ceph-mon[34645]: pgmap v611: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 210 op/s 2024-05-09T08:37:45.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 411812 7.77 MB/sec execute 241 sec latency 23.518 ms 2024-05-09T08:37:46.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 413177 7.76 MB/sec execute 242 sec latency 15.314 ms 2024-05-09T08:37:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:46 smithi160 ceph-mon[34070]: pgmap v612: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-09T08:37:47.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:46 smithi133 ceph-mon[29610]: pgmap v612: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-09T08:37:47.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:46 smithi188 ceph-mon[34645]: pgmap v612: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-05-09T08:37:47.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 414421 7.76 MB/sec execute 243 sec latency 23.313 ms 2024-05-09T08:37:48.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 415762 7.76 MB/sec execute 244 sec latency 18.075 ms 2024-05-09T08:37:49.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:48 smithi160 ceph-mon[34070]: pgmap v613: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:49.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:48 smithi133 ceph-mon[29610]: pgmap v613: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:49.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:48 smithi188 ceph-mon[34645]: pgmap v613: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-09T08:37:49.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 417418 7.76 MB/sec execute 245 sec latency 34.994 ms 2024-05-09T08:37:50.947 INFO:tasks.workunit.client.0.smithi133.stdout: 1 418819 7.77 MB/sec execute 246 sec latency 27.183 ms 2024-05-09T08:37:51.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:50 smithi160 ceph-mon[34070]: pgmap v614: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-05-09T08:37:51.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:50 smithi133 ceph-mon[29610]: pgmap v614: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-05-09T08:37:51.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:50 smithi188 ceph-mon[34645]: pgmap v614: 97 pgs: 97 active+clean; 161 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 194 op/s 2024-05-09T08:37:51.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 420089 7.76 MB/sec execute 247 sec latency 15.911 ms 2024-05-09T08:37:52.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 420967 7.74 MB/sec execute 248 sec latency 572.430 ms 2024-05-09T08:37:53.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:52 smithi160 ceph-mon[34070]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:37:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:52 smithi133 ceph-mon[29610]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:37:53.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:52 smithi188 ceph-mon[34645]: pgmap v615: 97 pgs: 97 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-05-09T08:37:53.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 422216 7.75 MB/sec execute 249 sec latency 27.392 ms 2024-05-09T08:37:54.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 423111 7.73 MB/sec execute 250 sec latency 24.934 ms 2024-05-09T08:37:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:54 smithi160 ceph-mon[34070]: pgmap v616: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-09T08:37:55.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:54 smithi133 ceph-mon[29610]: pgmap v616: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-09T08:37:55.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:54 smithi188 ceph-mon[34645]: pgmap v616: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-05-09T08:37:55.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 424637 7.73 MB/sec execute 251 sec latency 26.267 ms 2024-05-09T08:37:56.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 425772 7.73 MB/sec execute 252 sec latency 252.145 ms 2024-05-09T08:37:57.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:56 smithi160 ceph-mon[34070]: pgmap v617: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-05-09T08:37:57.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:56 smithi133 ceph-mon[29610]: pgmap v617: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-05-09T08:37:57.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:56 smithi188 ceph-mon[34645]: pgmap v617: 97 pgs: 97 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 182 op/s 2024-05-09T08:37:57.948 INFO:tasks.workunit.client.0.smithi133.stdout: 1 426840 7.72 MB/sec execute 253 sec latency 13.142 ms 2024-05-09T08:37:58.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 428179 7.71 MB/sec execute 254 sec latency 53.996 ms 2024-05-09T08:37:59.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:37:58 smithi160 ceph-mon[34070]: pgmap v618: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-09T08:37:59.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:37:58 smithi133 ceph-mon[29610]: pgmap v618: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-09T08:37:59.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:37:58 smithi188 ceph-mon[34645]: pgmap v618: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-05-09T08:37:59.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 429359 7.72 MB/sec execute 255 sec latency 235.128 ms 2024-05-09T08:38:00.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 430671 7.71 MB/sec execute 256 sec latency 17.772 ms 2024-05-09T08:38:01.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:00 smithi160 ceph-mon[34070]: pgmap v619: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:38:01.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:00 smithi133 ceph-mon[29610]: pgmap v619: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:38:01.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:00 smithi188 ceph-mon[34645]: pgmap v619: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-05-09T08:38:01.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 432095 7.71 MB/sec execute 257 sec latency 20.768 ms 2024-05-09T08:38:02.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 433625 7.71 MB/sec execute 258 sec latency 10.345 ms 2024-05-09T08:38:03.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:02 smithi160 ceph-mon[34070]: pgmap v620: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:03.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:02 smithi133 ceph-mon[29610]: pgmap v620: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:03.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:02 smithi188 ceph-mon[34645]: pgmap v620: 97 pgs: 97 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:03.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 435069 7.70 MB/sec execute 259 sec latency 454.226 ms 2024-05-09T08:38:04.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:03 smithi160 ceph-mon[34070]: pgmap v621: 97 pgs: 97 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-09T08:38:04.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:03 smithi133 ceph-mon[29610]: pgmap v621: 97 pgs: 97 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-09T08:38:04.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:03 smithi188 ceph-mon[34645]: pgmap v621: 97 pgs: 97 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-09T08:38:04.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 435336 7.68 MB/sec execute 260 sec latency 580.705 ms 2024-05-09T08:38:05.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 436973 7.69 MB/sec execute 261 sec latency 18.119 ms 2024-05-09T08:38:06.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:06 smithi160 ceph-mon[34070]: pgmap v622: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:06.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:06 smithi133 ceph-mon[29610]: pgmap v622: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:06.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:06 smithi188 ceph-mon[34645]: pgmap v622: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 189 op/s 2024-05-09T08:38:06.949 INFO:tasks.workunit.client.0.smithi133.stdout: 1 438751 7.69 MB/sec execute 262 sec latency 20.633 ms 2024-05-09T08:38:07.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 440226 7.71 MB/sec execute 263 sec latency 25.363 ms 2024-05-09T08:38:08.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:08 smithi160 ceph-mon[34070]: pgmap v623: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-05-09T08:38:08.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:08 smithi133 ceph-mon[29610]: pgmap v623: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-05-09T08:38:08.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:08 smithi188 ceph-mon[34645]: pgmap v623: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 241 op/s 2024-05-09T08:38:08.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 442165 7.70 MB/sec execute 264 sec latency 14.542 ms 2024-05-09T08:38:09.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 442734 7.69 MB/sec execute 265 sec latency 285.932 ms 2024-05-09T08:38:10.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:10 smithi160 ceph-mon[34070]: pgmap v624: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-05-09T08:38:10.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:10 smithi133 ceph-mon[29610]: pgmap v624: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-05-09T08:38:10.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:10 smithi188 ceph-mon[34645]: pgmap v624: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-05-09T08:38:10.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 444149 7.69 MB/sec execute 266 sec latency 14.543 ms 2024-05-09T08:38:11.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 445842 7.69 MB/sec execute 267 sec latency 27.105 ms 2024-05-09T08:38:12.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:12 smithi160 ceph-mon[34070]: pgmap v625: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-05-09T08:38:12.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:12 smithi133 ceph-mon[29610]: pgmap v625: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-05-09T08:38:12.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:12 smithi188 ceph-mon[34645]: pgmap v625: 97 pgs: 97 active+clean; 167 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-05-09T08:38:12.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 447234 7.70 MB/sec execute 268 sec latency 25.778 ms 2024-05-09T08:38:13.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 448614 7.70 MB/sec execute 269 sec latency 16.983 ms 2024-05-09T08:38:14.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:14 smithi160 ceph-mon[34070]: pgmap v626: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-09T08:38:14.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:14 smithi133 ceph-mon[29610]: pgmap v626: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-09T08:38:14.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:14 smithi188 ceph-mon[34645]: pgmap v626: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 254 op/s 2024-05-09T08:38:14.950 INFO:tasks.workunit.client.0.smithi133.stdout: 1 449840 7.69 MB/sec execute 270 sec latency 57.721 ms 2024-05-09T08:38:15.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 451183 7.69 MB/sec execute 271 sec latency 18.350 ms 2024-05-09T08:38:16.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:16 smithi160 ceph-mon[34070]: pgmap v627: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:16.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:16 smithi133 ceph-mon[29610]: pgmap v627: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:16.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:16 smithi188 ceph-mon[34645]: pgmap v627: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:16.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 452898 7.69 MB/sec execute 272 sec latency 25.739 ms 2024-05-09T08:38:17.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 453046 7.66 MB/sec execute 273 sec latency 696.666 ms 2024-05-09T08:38:18.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:18 smithi160 ceph-mon[34070]: pgmap v628: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-05-09T08:38:18.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:18 smithi133 ceph-mon[29610]: pgmap v628: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-05-09T08:38:18.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:18 smithi188 ceph-mon[34645]: pgmap v628: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-05-09T08:38:18.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 454467 7.68 MB/sec execute 274 sec latency 22.988 ms 2024-05-09T08:38:19.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 455932 7.67 MB/sec execute 275 sec latency 38.585 ms 2024-05-09T08:38:20.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:20 smithi160 ceph-mon[34070]: pgmap v629: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-09T08:38:20.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:20 smithi133 ceph-mon[29610]: pgmap v629: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-09T08:38:20.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:20 smithi188 ceph-mon[34645]: pgmap v629: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-05-09T08:38:20.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 456550 7.65 MB/sec execute 276 sec latency 330.266 ms 2024-05-09T08:38:21.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 458001 7.66 MB/sec execute 277 sec latency 23.505 ms 2024-05-09T08:38:22.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:22 smithi160 ceph-mon[34070]: pgmap v630: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:38:22.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:22 smithi133 ceph-mon[29610]: pgmap v630: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:38:22.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:22 smithi188 ceph-mon[34645]: pgmap v630: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-05-09T08:38:22.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 458821 7.70 MB/sec execute 278 sec latency 76.332 ms 2024-05-09T08:38:23.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 460750 7.71 MB/sec execute 279 sec latency 25.060 ms 2024-05-09T08:38:24.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:24 smithi160 ceph-mon[34070]: pgmap v631: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-09T08:38:24.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:24 smithi133 ceph-mon[29610]: pgmap v631: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-09T08:38:24.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:24 smithi188 ceph-mon[34645]: pgmap v631: 97 pgs: 97 active+clean; 173 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-09T08:38:24.951 INFO:tasks.workunit.client.0.smithi133.stdout: 1 462152 7.72 MB/sec execute 280 sec latency 21.937 ms 2024-05-09T08:38:25.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 463160 7.70 MB/sec execute 281 sec latency 47.430 ms 2024-05-09T08:38:26.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:26 smithi160 ceph-mon[34070]: pgmap v632: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:26.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:26 smithi133 ceph-mon[29610]: pgmap v632: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:26.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:26 smithi188 ceph-mon[34645]: pgmap v632: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-05-09T08:38:26.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 464661 7.71 MB/sec execute 282 sec latency 32.188 ms 2024-05-09T08:38:27.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 466134 7.71 MB/sec execute 283 sec latency 15.105 ms 2024-05-09T08:38:28.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:28 smithi160 ceph-mon[34070]: pgmap v633: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-05-09T08:38:28.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:28 smithi133 ceph-mon[29610]: pgmap v633: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-05-09T08:38:28.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:28 smithi188 ceph-mon[34645]: pgmap v633: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-05-09T08:38:28.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 467864 7.71 MB/sec execute 284 sec latency 37.735 ms 2024-05-09T08:38:29.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 469308 7.72 MB/sec execute 285 sec latency 24.730 ms 2024-05-09T08:38:30.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:30 smithi160 ceph-mon[34070]: pgmap v634: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-05-09T08:38:30.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:30 smithi133 ceph-mon[29610]: pgmap v634: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-05-09T08:38:30.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:30 smithi188 ceph-mon[34645]: pgmap v634: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-05-09T08:38:30.952 INFO:tasks.workunit.client.0.smithi133.stdout: 1 470602 7.71 MB/sec execute 286 sec latency 26.835 ms 2024-05-09T08:38:31.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 471932 7.71 MB/sec execute 287 sec latency 29.684 ms 2024-05-09T08:38:32.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:32 smithi160 ceph-mon[34070]: pgmap v635: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:38:32.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:32 smithi133 ceph-mon[29610]: pgmap v635: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:38:32.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:32 smithi188 ceph-mon[34645]: pgmap v635: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-09T08:38:32.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 473294 7.71 MB/sec execute 288 sec latency 12.286 ms 2024-05-09T08:38:33.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 474968 7.71 MB/sec execute 289 sec latency 29.495 ms 2024-05-09T08:38:34.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:34 smithi160 ceph-mon[34070]: pgmap v636: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:38:34.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:34 smithi133 ceph-mon[29610]: pgmap v636: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:38:34.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:34 smithi188 ceph-mon[34645]: pgmap v636: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-09T08:38:34.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 476351 7.72 MB/sec execute 290 sec latency 28.955 ms 2024-05-09T08:38:35.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 477523 7.71 MB/sec execute 291 sec latency 25.638 ms 2024-05-09T08:38:36.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:36 smithi160 ceph-mon[34070]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-09T08:38:36.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:36 smithi133 ceph-mon[29610]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-09T08:38:36.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:36 smithi188 ceph-mon[34645]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-05-09T08:38:36.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 478736 7.70 MB/sec execute 292 sec latency 29.290 ms 2024-05-09T08:38:37.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 480271 7.71 MB/sec execute 293 sec latency 13.256 ms 2024-05-09T08:38:38.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:38 smithi160 ceph-mon[34070]: pgmap v638: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:38:38.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:38 smithi133 ceph-mon[29610]: pgmap v638: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:38:38.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:38 smithi188 ceph-mon[34645]: pgmap v638: 97 pgs: 97 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:38:38.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 482059 7.71 MB/sec execute 294 sec latency 28.881 ms 2024-05-09T08:38:39.953 INFO:tasks.workunit.client.0.smithi133.stdout: 1 483495 7.72 MB/sec execute 295 sec latency 25.487 ms 2024-05-09T08:38:40.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:40 smithi160 ceph-mon[34070]: pgmap v639: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-05-09T08:38:40.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:38:40.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:40 smithi133 ceph-mon[29610]: pgmap v639: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-05-09T08:38:40.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:38:40.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:40 smithi188 ceph-mon[34645]: pgmap v639: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-05-09T08:38:40.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:38:40.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 484707 7.71 MB/sec execute 296 sec latency 42.378 ms 2024-05-09T08:38:41.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:38:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:38:41.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:38:41.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:38:41.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:38:41.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:38:41.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:38:41.431 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:38:41.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 486062 7.71 MB/sec execute 297 sec latency 32.992 ms 2024-05-09T08:38:42.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:42 smithi160 ceph-mon[34070]: pgmap v640: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-05-09T08:38:42.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:42 smithi133 ceph-mon[29610]: pgmap v640: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-05-09T08:38:42.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:42 smithi188 ceph-mon[34645]: pgmap v640: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.1 MiB/s wr, 166 op/s 2024-05-09T08:38:42.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 487363 7.71 MB/sec execute 298 sec latency 10.702 ms 2024-05-09T08:38:43.954 INFO:tasks.workunit.client.0.smithi133.stdout: 1 489151 7.70 MB/sec execute 299 sec latency 30.997 ms 2024-05-09T08:38:44.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:44 smithi160 ceph-mon[34070]: pgmap v641: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:38:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:44 smithi133 ceph-mon[29610]: pgmap v641: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:38:44.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:44 smithi188 ceph-mon[34645]: pgmap v641: 97 pgs: 97 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:38:44.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 490439 7.71 MB/sec execute 300 sec latency 26.476 ms 2024-05-09T08:38:45.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 491444 7.70 MB/sec execute 301 sec latency 66.123 ms 2024-05-09T08:38:46.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:46 smithi160 ceph-mon[34070]: pgmap v642: 97 pgs: 97 active+clean; 164 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-09T08:38:46.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:46 smithi133 ceph-mon[29610]: pgmap v642: 97 pgs: 97 active+clean; 164 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-09T08:38:46.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:46 smithi188 ceph-mon[34645]: pgmap v642: 97 pgs: 97 active+clean; 164 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-05-09T08:38:46.955 INFO:tasks.workunit.client.0.smithi133.stdout: 1 492885 7.69 MB/sec execute 302 sec latency 49.021 ms 2024-05-09T08:38:47.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 494278 7.71 MB/sec execute 303 sec latency 22.969 ms 2024-05-09T08:38:48.729 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:48 smithi133 ceph-mon[29610]: pgmap v643: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:38:48.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:48 smithi160 ceph-mon[34070]: pgmap v643: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:38:48.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:48 smithi188 ceph-mon[34645]: pgmap v643: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:38:48.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 495943 7.70 MB/sec execute 304 sec latency 13.639 ms 2024-05-09T08:38:49.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 497214 7.70 MB/sec execute 305 sec latency 27.794 ms 2024-05-09T08:38:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:50 smithi160 ceph-mon[34070]: pgmap v644: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-05-09T08:38:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:50 smithi133 ceph-mon[29610]: pgmap v644: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-05-09T08:38:50.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:50 smithi188 ceph-mon[34645]: pgmap v644: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-05-09T08:38:50.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 498313 7.69 MB/sec execute 306 sec latency 47.869 ms 2024-05-09T08:38:51.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 499857 7.69 MB/sec execute 307 sec latency 49.947 ms 2024-05-09T08:38:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:52 smithi160 ceph-mon[34070]: pgmap v645: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-05-09T08:38:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:52 smithi133 ceph-mon[29610]: pgmap v645: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-05-09T08:38:52.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:52 smithi188 ceph-mon[34645]: pgmap v645: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-05-09T08:38:52.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 501313 7.70 MB/sec execute 308 sec latency 25.183 ms 2024-05-09T08:38:53.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 503082 7.70 MB/sec execute 309 sec latency 16.458 ms 2024-05-09T08:38:54.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:54 smithi160 ceph-mon[34070]: pgmap v646: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-05-09T08:38:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:54 smithi133 ceph-mon[29610]: pgmap v646: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-05-09T08:38:54.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:54 smithi188 ceph-mon[34645]: pgmap v646: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 1.9 MiB/s wr, 162 op/s 2024-05-09T08:38:54.956 INFO:tasks.workunit.client.0.smithi133.stdout: 1 504496 7.71 MB/sec execute 310 sec latency 24.696 ms 2024-05-09T08:38:55.957 INFO:tasks.workunit.client.0.smithi133.stdout: 1 505675 7.70 MB/sec execute 311 sec latency 94.139 ms 2024-05-09T08:38:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:56 smithi160 ceph-mon[34070]: pgmap v647: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-09T08:38:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:56 smithi133 ceph-mon[29610]: pgmap v647: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-09T08:38:56.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:56 smithi188 ceph-mon[34645]: pgmap v647: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.9 MiB/s wr, 174 op/s 2024-05-09T08:38:56.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 507252 7.70 MB/sec execute 312 sec latency 28.543 ms 2024-05-09T08:38:57.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 508745 7.70 MB/sec execute 313 sec latency 11.800 ms 2024-05-09T08:38:58.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:38:58 smithi160 ceph-mon[34070]: pgmap v648: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-09T08:38:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:38:58 smithi133 ceph-mon[29610]: pgmap v648: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-09T08:38:58.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:38:58 smithi188 ceph-mon[34645]: pgmap v648: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 2.2 MiB/s wr, 149 op/s 2024-05-09T08:38:58.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 510470 7.70 MB/sec execute 314 sec latency 29.013 ms 2024-05-09T08:38:59.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 511953 7.72 MB/sec execute 315 sec latency 21.492 ms 2024-05-09T08:39:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:00 smithi160 ceph-mon[34070]: pgmap v649: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-05-09T08:39:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:00 smithi133 ceph-mon[29610]: pgmap v649: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-05-09T08:39:00.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:00 smithi188 ceph-mon[34645]: pgmap v649: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.7 MiB/s wr, 142 op/s 2024-05-09T08:39:00.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 513454 7.71 MB/sec execute 316 sec latency 38.652 ms 2024-05-09T08:39:01.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 515123 7.72 MB/sec execute 317 sec latency 23.732 ms 2024-05-09T08:39:02.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:02 smithi160 ceph-mon[34070]: pgmap v650: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-05-09T08:39:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:02 smithi133 ceph-mon[29610]: pgmap v650: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-05-09T08:39:02.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:02 smithi188 ceph-mon[34645]: pgmap v650: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-05-09T08:39:02.958 INFO:tasks.workunit.client.0.smithi133.stdout: 1 516549 7.72 MB/sec execute 318 sec latency 12.822 ms 2024-05-09T08:39:03.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 518135 7.72 MB/sec execute 319 sec latency 28.934 ms 2024-05-09T08:39:04.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:04 smithi160 ceph-mon[34070]: pgmap v651: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.5 MiB/s wr, 153 op/s 2024-05-09T08:39:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:04 smithi133 ceph-mon[29610]: pgmap v651: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.5 MiB/s wr, 153 op/s 2024-05-09T08:39:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:04 smithi188 ceph-mon[34645]: pgmap v651: 97 pgs: 97 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 3.5 MiB/s wr, 153 op/s 2024-05-09T08:39:04.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 519556 7.72 MB/sec execute 320 sec latency 15.648 ms 2024-05-09T08:39:05.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 521170 7.71 MB/sec execute 321 sec latency 36.718 ms 2024-05-09T08:39:06.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:06 smithi160 ceph-mon[34070]: pgmap v652: 97 pgs: 97 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-05-09T08:39:06.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:06 smithi133 ceph-mon[29610]: pgmap v652: 97 pgs: 97 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-05-09T08:39:06.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:06 smithi188 ceph-mon[34645]: pgmap v652: 97 pgs: 97 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-05-09T08:39:06.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 522811 7.73 MB/sec execute 322 sec latency 46.442 ms 2024-05-09T08:39:07.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 524652 7.72 MB/sec execute 323 sec latency 23.642 ms 2024-05-09T08:39:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:08 smithi160 ceph-mon[34070]: pgmap v653: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.6 MiB/s wr, 152 op/s 2024-05-09T08:39:08.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:08 smithi133 ceph-mon[29610]: pgmap v653: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.6 MiB/s wr, 152 op/s 2024-05-09T08:39:08.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:08 smithi188 ceph-mon[34645]: pgmap v653: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.6 MiB/s wr, 152 op/s 2024-05-09T08:39:08.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 526211 7.74 MB/sec execute 324 sec latency 21.906 ms 2024-05-09T08:39:09.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 528159 7.74 MB/sec execute 325 sec latency 34.237 ms 2024-05-09T08:39:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:10 smithi160 ceph-mon[34070]: pgmap v654: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-05-09T08:39:10.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:10 smithi133 ceph-mon[29610]: pgmap v654: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-05-09T08:39:10.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:10 smithi188 ceph-mon[34645]: pgmap v654: 97 pgs: 97 active+clean; 196 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 3.0 MiB/s wr, 148 op/s 2024-05-09T08:39:10.959 INFO:tasks.workunit.client.0.smithi133.stdout: 1 529633 7.75 MB/sec execute 326 sec latency 24.788 ms 2024-05-09T08:39:11.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 531002 7.74 MB/sec execute 327 sec latency 30.939 ms 2024-05-09T08:39:12.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:12 smithi160 ceph-mon[34070]: pgmap v655: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:39:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:12 smithi133 ceph-mon[29610]: pgmap v655: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:39:12.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:12 smithi188 ceph-mon[34645]: pgmap v655: 97 pgs: 97 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:39:12.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 532546 7.74 MB/sec execute 328 sec latency 28.310 ms 2024-05-09T08:39:13.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 533716 7.74 MB/sec execute 329 sec latency 17.053 ms 2024-05-09T08:39:14.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:14 smithi160 ceph-mon[34070]: pgmap v656: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:39:14.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:14 smithi133 ceph-mon[29610]: pgmap v656: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:39:14.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:14 smithi188 ceph-mon[34645]: pgmap v656: 97 pgs: 97 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:39:14.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 535342 7.73 MB/sec execute 330 sec latency 26.677 ms 2024-05-09T08:39:15.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 536801 7.75 MB/sec execute 331 sec latency 22.293 ms 2024-05-09T08:39:16.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:16 smithi160 ceph-mon[34070]: pgmap v657: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-05-09T08:39:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:16 smithi133 ceph-mon[29610]: pgmap v657: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-05-09T08:39:16.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:16 smithi188 ceph-mon[34645]: pgmap v657: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-05-09T08:39:16.960 INFO:tasks.workunit.client.0.smithi133.stdout: 1 538079 7.74 MB/sec execute 332 sec latency 21.604 ms 2024-05-09T08:39:17.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 539451 7.74 MB/sec execute 333 sec latency 26.097 ms 2024-05-09T08:39:18.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:18 smithi160 ceph-mon[34070]: pgmap v658: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:39:18.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:18 smithi133 ceph-mon[29610]: pgmap v658: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:39:18.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:18 smithi188 ceph-mon[34645]: pgmap v658: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:39:18.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 540780 7.74 MB/sec execute 334 sec latency 13.950 ms 2024-05-09T08:39:19.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 542428 7.73 MB/sec execute 335 sec latency 29.439 ms 2024-05-09T08:39:20.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:20 smithi160 ceph-mon[34070]: pgmap v659: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-09T08:39:20.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:20 smithi133 ceph-mon[29610]: pgmap v659: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-09T08:39:20.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:20 smithi188 ceph-mon[34645]: pgmap v659: 97 pgs: 97 active+clean; 200 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.7 MiB/s wr, 154 op/s 2024-05-09T08:39:20.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 543845 7.74 MB/sec execute 336 sec latency 26.605 ms 2024-05-09T08:39:21.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 545129 7.74 MB/sec execute 337 sec latency 32.803 ms 2024-05-09T08:39:22.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:22 smithi160 ceph-mon[34070]: pgmap v660: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:39:22.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:22 smithi133 ceph-mon[29610]: pgmap v660: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:39:22.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:22 smithi188 ceph-mon[34645]: pgmap v660: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-05-09T08:39:22.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 546790 7.74 MB/sec execute 338 sec latency 28.209 ms 2024-05-09T08:39:23.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 547960 7.74 MB/sec execute 339 sec latency 17.640 ms 2024-05-09T08:39:24.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:24 smithi160 ceph-mon[34070]: pgmap v661: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-05-09T08:39:24.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:24 smithi133 ceph-mon[29610]: pgmap v661: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-05-09T08:39:24.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:24 smithi188 ceph-mon[34645]: pgmap v661: 97 pgs: 97 active+clean; 202 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-05-09T08:39:24.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 549562 7.73 MB/sec execute 340 sec latency 29.682 ms 2024-05-09T08:39:25.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 551014 7.74 MB/sec execute 341 sec latency 21.349 ms 2024-05-09T08:39:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:26 smithi160 ceph-mon[34070]: pgmap v662: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:39:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:26 smithi133 ceph-mon[29610]: pgmap v662: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:39:26.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:26 smithi188 ceph-mon[34645]: pgmap v662: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:39:26.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 552593 7.74 MB/sec execute 342 sec latency 57.426 ms 2024-05-09T08:39:27.961 INFO:tasks.workunit.client.0.smithi133.stdout: 1 554123 7.75 MB/sec execute 343 sec latency 61.631 ms 2024-05-09T08:39:28.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:28 smithi160 ceph-mon[34070]: pgmap v663: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-09T08:39:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:28 smithi133 ceph-mon[29610]: pgmap v663: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-09T08:39:28.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:28 smithi188 ceph-mon[34645]: pgmap v663: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.8 MiB/s wr, 145 op/s 2024-05-09T08:39:28.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 555163 7.73 MB/sec execute 344 sec latency 15.015 ms 2024-05-09T08:39:29.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 556720 7.73 MB/sec execute 345 sec latency 26.912 ms 2024-05-09T08:39:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:30 smithi160 ceph-mon[34070]: pgmap v664: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:39:30.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:30 smithi133 ceph-mon[29610]: pgmap v664: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:39:30.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:30 smithi188 ceph-mon[34645]: pgmap v664: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:39:30.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 558177 7.74 MB/sec execute 346 sec latency 22.929 ms 2024-05-09T08:39:31.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 560001 7.74 MB/sec execute 347 sec latency 25.688 ms 2024-05-09T08:39:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:32 smithi160 ceph-mon[34070]: pgmap v665: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-09T08:39:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:32 smithi133 ceph-mon[29610]: pgmap v665: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-09T08:39:32.934 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:32 smithi188 ceph-mon[34645]: pgmap v665: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-05-09T08:39:32.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 561246 7.75 MB/sec execute 348 sec latency 31.928 ms 2024-05-09T08:39:33.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 562243 7.73 MB/sec execute 349 sec latency 19.770 ms 2024-05-09T08:39:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:34 smithi160 ceph-mon[34070]: pgmap v666: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:34 smithi133 ceph-mon[29610]: pgmap v666: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:34.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:34 smithi188 ceph-mon[34645]: pgmap v666: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:34.962 INFO:tasks.workunit.client.0.smithi133.stdout: 1 563722 7.73 MB/sec execute 350 sec latency 43.048 ms 2024-05-09T08:39:35.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 565158 7.74 MB/sec execute 351 sec latency 26.168 ms 2024-05-09T08:39:36.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:36 smithi160 ceph-mon[34070]: pgmap v667: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:39:36.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:36 smithi133 ceph-mon[29610]: pgmap v667: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:39:36.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:36 smithi188 ceph-mon[34645]: pgmap v667: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:39:36.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 566413 7.73 MB/sec execute 352 sec latency 32.894 ms 2024-05-09T08:39:37.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 567776 7.73 MB/sec execute 353 sec latency 24.325 ms 2024-05-09T08:39:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:38 smithi160 ceph-mon[34070]: pgmap v668: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:38 smithi133 ceph-mon[29610]: pgmap v668: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:38.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:38 smithi188 ceph-mon[34645]: pgmap v668: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:39:38.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 569067 7.73 MB/sec execute 354 sec latency 18.166 ms 2024-05-09T08:39:39.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 570745 7.73 MB/sec execute 355 sec latency 32.026 ms 2024-05-09T08:39:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:40 smithi160 ceph-mon[34070]: pgmap v669: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-09T08:39:40.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:40 smithi133 ceph-mon[29610]: pgmap v669: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-09T08:39:40.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:40 smithi188 ceph-mon[34645]: pgmap v669: 97 pgs: 97 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-05-09T08:39:40.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 572025 7.74 MB/sec execute 356 sec latency 24.133 ms 2024-05-09T08:39:41.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:41 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:39:41.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:41 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:39:41.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:41 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:39:41.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 573312 7.73 MB/sec execute 357 sec latency 24.418 ms 2024-05-09T08:39:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:42 smithi160 ceph-mon[34070]: pgmap v670: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-05-09T08:39:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:39:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:39:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:39:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:42 smithi133 ceph-mon[29610]: pgmap v670: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-05-09T08:39:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:39:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:39:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:39:42.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:42 smithi188 ceph-mon[34645]: pgmap v670: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-05-09T08:39:42.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:39:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:39:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:39:42.963 INFO:tasks.workunit.client.0.smithi133.stdout: 1 574653 7.73 MB/sec execute 358 sec latency 25.799 ms 2024-05-09T08:39:43.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 576019 7.73 MB/sec execute 359 sec latency 14.707 ms 2024-05-09T08:39:44.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:44 smithi160 ceph-mon[34070]: pgmap v671: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:39:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:44 smithi133 ceph-mon[29610]: pgmap v671: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:39:44.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:44 smithi188 ceph-mon[34645]: pgmap v671: 97 pgs: 97 active+clean; 210 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:39:44.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 577824 7.73 MB/sec execute 360 sec latency 24.961 ms 2024-05-09T08:39:45.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 579194 7.74 MB/sec execute 361 sec latency 23.611 ms 2024-05-09T08:39:46.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:46 smithi160 ceph-mon[34070]: pgmap v672: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-05-09T08:39:46.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:46 smithi133 ceph-mon[29610]: pgmap v672: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-05-09T08:39:46.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:46 smithi188 ceph-mon[34645]: pgmap v672: 97 pgs: 97 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-05-09T08:39:46.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 580160 7.72 MB/sec execute 362 sec latency 33.711 ms 2024-05-09T08:39:47.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 581573 7.72 MB/sec execute 363 sec latency 25.244 ms 2024-05-09T08:39:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:48 smithi160 ceph-mon[34070]: pgmap v673: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:39:48.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:48 smithi133 ceph-mon[29610]: pgmap v673: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:39:48.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:48 smithi188 ceph-mon[34645]: pgmap v673: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:39:48.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 583004 7.73 MB/sec execute 364 sec latency 15.702 ms 2024-05-09T08:39:49.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 584827 7.73 MB/sec execute 365 sec latency 23.065 ms 2024-05-09T08:39:50.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:50 smithi160 ceph-mon[34070]: pgmap v674: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-05-09T08:39:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:50 smithi133 ceph-mon[29610]: pgmap v674: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-05-09T08:39:50.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:50 smithi188 ceph-mon[34645]: pgmap v674: 97 pgs: 97 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-05-09T08:39:50.964 INFO:tasks.workunit.client.0.smithi133.stdout: 1 586075 7.73 MB/sec execute 366 sec latency 25.540 ms 2024-05-09T08:39:51.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 587226 7.72 MB/sec execute 367 sec latency 22.502 ms 2024-05-09T08:39:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:52 smithi160 ceph-mon[34070]: pgmap v675: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-09T08:39:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:52 smithi133 ceph-mon[29610]: pgmap v675: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-09T08:39:52.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:52 smithi188 ceph-mon[34645]: pgmap v675: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-05-09T08:39:52.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 588680 7.72 MB/sec execute 368 sec latency 24.573 ms 2024-05-09T08:39:53.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 590401 7.73 MB/sec execute 369 sec latency 18.909 ms 2024-05-09T08:39:54.911 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:54 smithi160 ceph-mon[34070]: pgmap v676: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:39:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:54 smithi133 ceph-mon[29610]: pgmap v676: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:39:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:54 smithi188 ceph-mon[34645]: pgmap v676: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:39:54.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 592113 7.73 MB/sec execute 370 sec latency 25.523 ms 2024-05-09T08:39:55.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 593707 7.74 MB/sec execute 371 sec latency 18.829 ms 2024-05-09T08:39:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:56 smithi160 ceph-mon[34070]: pgmap v677: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-05-09T08:39:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:56 smithi133 ceph-mon[29610]: pgmap v677: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-05-09T08:39:56.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:56 smithi188 ceph-mon[34645]: pgmap v677: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-05-09T08:39:56.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 595573 7.73 MB/sec execute 372 sec latency 25.197 ms 2024-05-09T08:39:57.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 597092 7.75 MB/sec execute 373 sec latency 48.085 ms 2024-05-09T08:39:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:39:58 smithi160 ceph-mon[34070]: pgmap v678: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-09T08:39:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:39:58 smithi133 ceph-mon[29610]: pgmap v678: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-09T08:39:58.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:39:58 smithi188 ceph-mon[34645]: pgmap v678: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-05-09T08:39:58.965 INFO:tasks.workunit.client.0.smithi133.stdout: 1 599040 7.74 MB/sec execute 374 sec latency 21.288 ms 2024-05-09T08:39:59.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 600519 7.76 MB/sec execute 375 sec latency 19.177 ms 2024-05-09T08:40:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:00 smithi160 ceph-mon[34070]: pgmap v679: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-05-09T08:40:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:00 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:40:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:00 smithi133 ceph-mon[29610]: pgmap v679: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-05-09T08:40:00.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:00 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:40:00.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:00 smithi188 ceph-mon[34645]: pgmap v679: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.8 MiB/s wr, 135 op/s 2024-05-09T08:40:00.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:00 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:40:00.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 602145 7.75 MB/sec execute 376 sec latency 13.983 ms 2024-05-09T08:40:01.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 603641 7.75 MB/sec execute 377 sec latency 25.707 ms 2024-05-09T08:40:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:02 smithi160 ceph-mon[34070]: pgmap v680: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-09T08:40:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:02 smithi133 ceph-mon[29610]: pgmap v680: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-09T08:40:02.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:02 smithi188 ceph-mon[34645]: pgmap v680: 97 pgs: 97 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-05-09T08:40:02.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 604806 7.75 MB/sec execute 378 sec latency 22.679 ms 2024-05-09T08:40:03.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 606363 7.75 MB/sec execute 379 sec latency 46.201 ms 2024-05-09T08:40:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:04 smithi133 ceph-mon[29610]: pgmap v681: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-05-09T08:40:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:04 smithi188 ceph-mon[34645]: pgmap v681: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-05-09T08:40:04.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 607870 7.75 MB/sec execute 380 sec latency 21.953 ms 2024-05-09T08:40:05.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:04 smithi160 ceph-mon[34070]: pgmap v681: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-05-09T08:40:05.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 609714 7.75 MB/sec execute 381 sec latency 25.530 ms 2024-05-09T08:40:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:06 smithi133 ceph-mon[29610]: pgmap v682: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-09T08:40:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:06 smithi188 ceph-mon[34645]: pgmap v682: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-09T08:40:06.966 INFO:tasks.workunit.client.0.smithi133.stdout: 1 611109 7.76 MB/sec execute 382 sec latency 22.513 ms 2024-05-09T08:40:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:06 smithi160 ceph-mon[34070]: pgmap v682: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-05-09T08:40:07.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 612304 7.76 MB/sec execute 383 sec latency 35.506 ms 2024-05-09T08:40:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:08 smithi133 ceph-mon[29610]: pgmap v683: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-05-09T08:40:08.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:08 smithi188 ceph-mon[34645]: pgmap v683: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-05-09T08:40:08.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 613792 7.75 MB/sec execute 384 sec latency 31.988 ms 2024-05-09T08:40:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:08 smithi160 ceph-mon[34070]: pgmap v683: 97 pgs: 97 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 2.0 MiB/s wr, 179 op/s 2024-05-09T08:40:09.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 615097 7.75 MB/sec execute 385 sec latency 13.424 ms 2024-05-09T08:40:10.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:10 smithi188 ceph-mon[34645]: pgmap v684: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-09T08:40:10.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 616880 7.75 MB/sec execute 386 sec latency 24.562 ms 2024-05-09T08:40:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:10 smithi160 ceph-mon[34070]: pgmap v684: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-09T08:40:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:10 smithi133 ceph-mon[29610]: pgmap v684: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-05-09T08:40:11.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 618464 7.76 MB/sec execute 387 sec latency 20.264 ms 2024-05-09T08:40:12.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 620330 7.76 MB/sec execute 388 sec latency 49.872 ms 2024-05-09T08:40:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:12 smithi160 ceph-mon[34070]: pgmap v685: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:13.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:12 smithi133 ceph-mon[29610]: pgmap v685: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:13.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:12 smithi188 ceph-mon[34645]: pgmap v685: 97 pgs: 97 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:13.967 INFO:tasks.workunit.client.0.smithi133.stdout: 1 621815 7.77 MB/sec execute 389 sec latency 47.698 ms 2024-05-09T08:40:14.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 623562 7.77 MB/sec execute 390 sec latency 12.304 ms 2024-05-09T08:40:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:14 smithi160 ceph-mon[34070]: pgmap v686: 97 pgs: 97 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:40:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:14 smithi133 ceph-mon[29610]: pgmap v686: 97 pgs: 97 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:40:15.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:14 smithi188 ceph-mon[34645]: pgmap v686: 97 pgs: 97 active+clean; 224 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:40:15.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 625179 7.78 MB/sec execute 391 sec latency 23.706 ms 2024-05-09T08:40:16.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 626735 7.77 MB/sec execute 392 sec latency 10.663 ms 2024-05-09T08:40:17.161 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:16 smithi160 ceph-mon[34070]: pgmap v687: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:16 smithi133 ceph-mon[29610]: pgmap v687: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:17.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:16 smithi188 ceph-mon[34645]: pgmap v687: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:17.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 628437 7.78 MB/sec execute 393 sec latency 48.626 ms 2024-05-09T08:40:18.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:18 smithi160 ceph-mon[34070]: pgmap v688: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-05-09T08:40:18.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 629675 7.77 MB/sec execute 394 sec latency 18.282 ms 2024-05-09T08:40:18.979 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:18 smithi133 ceph-mon[29610]: pgmap v688: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-05-09T08:40:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:18 smithi188 ceph-mon[34645]: pgmap v688: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-05-09T08:40:19.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 631200 7.77 MB/sec execute 395 sec latency 24.982 ms 2024-05-09T08:40:20.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 632712 7.78 MB/sec execute 396 sec latency 21.955 ms 2024-05-09T08:40:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:20 smithi160 ceph-mon[34070]: pgmap v689: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:21.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:20 smithi133 ceph-mon[29610]: pgmap v689: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:21.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:20 smithi188 ceph-mon[34645]: pgmap v689: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-05-09T08:40:21.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 634551 7.78 MB/sec execute 397 sec latency 27.833 ms 2024-05-09T08:40:22.968 INFO:tasks.workunit.client.0.smithi133.stdout: 1 635818 7.78 MB/sec execute 398 sec latency 25.359 ms 2024-05-09T08:40:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:22 smithi160 ceph-mon[34070]: pgmap v690: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-05-09T08:40:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:22 smithi133 ceph-mon[29610]: pgmap v690: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-05-09T08:40:23.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:22 smithi188 ceph-mon[34645]: pgmap v690: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-05-09T08:40:23.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 636781 7.77 MB/sec execute 399 sec latency 21.150 ms 2024-05-09T08:40:24.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 638207 7.77 MB/sec execute 400 sec latency 26.401 ms 2024-05-09T08:40:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:24 smithi160 ceph-mon[34070]: pgmap v691: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-05-09T08:40:25.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:24 smithi133 ceph-mon[29610]: pgmap v691: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-05-09T08:40:25.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:24 smithi188 ceph-mon[34645]: pgmap v691: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-05-09T08:40:25.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 639649 7.78 MB/sec execute 401 sec latency 25.634 ms 2024-05-09T08:40:26.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 641548 7.78 MB/sec execute 402 sec latency 15.282 ms 2024-05-09T08:40:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:26 smithi160 ceph-mon[34070]: pgmap v692: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-05-09T08:40:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:26 smithi133 ceph-mon[29610]: pgmap v692: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-05-09T08:40:27.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:26 smithi188 ceph-mon[34645]: pgmap v692: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-05-09T08:40:27.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 642626 7.77 MB/sec execute 403 sec latency 32.016 ms 2024-05-09T08:40:28.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 643612 7.77 MB/sec execute 404 sec latency 22.019 ms 2024-05-09T08:40:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:28 smithi160 ceph-mon[34070]: pgmap v693: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:40:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:28 smithi133 ceph-mon[29610]: pgmap v693: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:40:29.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:28 smithi188 ceph-mon[34645]: pgmap v693: 97 pgs: 97 active+clean; 231 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:40:29.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 645230 7.77 MB/sec execute 405 sec latency 48.377 ms 2024-05-09T08:40:30.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 646565 7.78 MB/sec execute 406 sec latency 22.515 ms 2024-05-09T08:40:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:30 smithi160 ceph-mon[34070]: pgmap v694: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:31.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:30 smithi133 ceph-mon[29610]: pgmap v694: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:31.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:30 smithi188 ceph-mon[34645]: pgmap v694: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-05-09T08:40:31.969 INFO:tasks.workunit.client.0.smithi133.stdout: 1 647522 7.76 MB/sec execute 407 sec latency 20.016 ms 2024-05-09T08:40:32.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 648932 7.76 MB/sec execute 408 sec latency 21.666 ms 2024-05-09T08:40:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:32 smithi160 ceph-mon[34070]: pgmap v695: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-05-09T08:40:33.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:32 smithi133 ceph-mon[29610]: pgmap v695: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-05-09T08:40:33.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:32 smithi188 ceph-mon[34645]: pgmap v695: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-05-09T08:40:33.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 650322 7.77 MB/sec execute 409 sec latency 41.031 ms 2024-05-09T08:40:34.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 651655 7.76 MB/sec execute 410 sec latency 22.079 ms 2024-05-09T08:40:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:34 smithi160 ceph-mon[34070]: pgmap v696: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-05-09T08:40:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:34 smithi133 ceph-mon[29610]: pgmap v696: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-05-09T08:40:35.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:34 smithi188 ceph-mon[34645]: pgmap v696: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-05-09T08:40:35.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 652882 7.76 MB/sec execute 411 sec latency 49.347 ms 2024-05-09T08:40:36.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 654101 7.76 MB/sec execute 412 sec latency 16.837 ms 2024-05-09T08:40:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:36 smithi160 ceph-mon[34070]: pgmap v697: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-09T08:40:37.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:36 smithi133 ceph-mon[29610]: pgmap v697: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-09T08:40:37.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:36 smithi188 ceph-mon[34645]: pgmap v697: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-05-09T08:40:37.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 655799 7.75 MB/sec execute 413 sec latency 16.870 ms 2024-05-09T08:40:38.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 656813 7.75 MB/sec execute 414 sec latency 78.740 ms 2024-05-09T08:40:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:38 smithi160 ceph-mon[34070]: pgmap v698: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:40:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:38 smithi133 ceph-mon[29610]: pgmap v698: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:40:39.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:38 smithi188 ceph-mon[34645]: pgmap v698: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:40:39.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 657895 7.75 MB/sec execute 415 sec latency 30.395 ms 2024-05-09T08:40:40.970 INFO:tasks.workunit.client.0.smithi133.stdout: 1 659439 7.74 MB/sec execute 416 sec latency 28.546 ms 2024-05-09T08:40:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:40 smithi160 ceph-mon[34070]: pgmap v699: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-05-09T08:40:41.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:40 smithi133 ceph-mon[29610]: pgmap v699: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-05-09T08:40:41.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:40 smithi188 ceph-mon[34645]: pgmap v699: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-05-09T08:40:41.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 660838 7.75 MB/sec execute 417 sec latency 21.657 ms 2024-05-09T08:40:42.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 662225 7.75 MB/sec execute 418 sec latency 11.027 ms 2024-05-09T08:40:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:42 smithi160 ceph-mon[34070]: pgmap v700: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-05-09T08:40:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:42 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:40:43.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:42 smithi133 ceph-mon[29610]: pgmap v700: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-05-09T08:40:43.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:42 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:40:43.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:42 smithi188 ceph-mon[34645]: pgmap v700: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-05-09T08:40:43.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:42 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:40:43.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 663906 7.75 MB/sec execute 419 sec latency 57.583 ms 2024-05-09T08:40:44.586 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: pgmap v701: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.587 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:44 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: pgmap v701: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:44 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: pgmap v701: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.9 MiB/s wr, 163 op/s 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:44 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:44.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 665010 7.75 MB/sec execute 420 sec latency 20.707 ms 2024-05-09T08:40:45.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 666987 7.76 MB/sec execute 421 sec latency 18.607 ms 2024-05-09T08:40:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:46 smithi160 ceph-mon[34070]: pgmap v702: 97 pgs: 97 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:40:46.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:46 smithi133 ceph-mon[29610]: pgmap v702: 97 pgs: 97 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:40:46.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:46 smithi188 ceph-mon[34645]: pgmap v702: 97 pgs: 97 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-05-09T08:40:46.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 668407 7.75 MB/sec execute 422 sec latency 21.578 ms 2024-05-09T08:40:47.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:47 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:47 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:47 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:47.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 670113 7.75 MB/sec execute 423 sec latency 20.490 ms 2024-05-09T08:40:48.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:48 smithi160 ceph-mon[34070]: pgmap v703: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:40:48.562 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:48 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:40:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:48 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:40:48.563 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:48 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:48.579 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:48 smithi133 ceph-mon[29610]: pgmap v703: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:40:48.579 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:48 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:40:48.579 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:48 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:40:48.579 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:48 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:48.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:48 smithi188 ceph-mon[34645]: pgmap v703: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:40:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:48 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:40:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:48 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:40:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:48 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:40:48.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 671419 7.76 MB/sec execute 424 sec latency 59.280 ms 2024-05-09T08:40:49.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 673358 7.76 MB/sec execute 425 sec latency 15.425 ms 2024-05-09T08:40:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:50 smithi160 ceph-mon[34070]: pgmap v704: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:40:50.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:50 smithi133 ceph-mon[29610]: pgmap v704: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:40:50.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:50 smithi188 ceph-mon[34645]: pgmap v704: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:40:50.971 INFO:tasks.workunit.client.0.smithi133.stdout: 1 674835 7.77 MB/sec execute 426 sec latency 60.107 ms 2024-05-09T08:40:51.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 676192 7.76 MB/sec execute 427 sec latency 19.304 ms 2024-05-09T08:40:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:52 smithi160 ceph-mon[34070]: pgmap v705: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-05-09T08:40:52.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:52 smithi133 ceph-mon[29610]: pgmap v705: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-05-09T08:40:52.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:52 smithi188 ceph-mon[34645]: pgmap v705: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2024-05-09T08:40:52.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 677968 7.76 MB/sec execute 428 sec latency 22.294 ms 2024-05-09T08:40:53.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 679278 7.76 MB/sec execute 429 sec latency 35.903 ms 2024-05-09T08:40:54.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:54 smithi160 ceph-mon[34070]: pgmap v706: 97 pgs: 97 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:40:54.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:54 smithi133 ceph-mon[29610]: pgmap v706: 97 pgs: 97 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:40:54.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:54 smithi188 ceph-mon[34645]: pgmap v706: 97 pgs: 97 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:40:54.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 680913 7.76 MB/sec execute 430 sec latency 25.360 ms 2024-05-09T08:40:55.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 682614 7.77 MB/sec execute 431 sec latency 14.407 ms 2024-05-09T08:40:56.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:56 smithi160 ceph-mon[34070]: pgmap v707: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-09T08:40:56.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:56 smithi133 ceph-mon[29610]: pgmap v707: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-09T08:40:56.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:56 smithi188 ceph-mon[34645]: pgmap v707: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-05-09T08:40:56.972 INFO:tasks.workunit.client.0.smithi133.stdout: 1 684411 7.77 MB/sec execute 432 sec latency 23.863 ms 2024-05-09T08:40:57.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 685909 7.78 MB/sec execute 433 sec latency 21.845 ms 2024-05-09T08:40:58.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:40:58 smithi160 ceph-mon[34070]: pgmap v708: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-05-09T08:40:58.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:40:58 smithi133 ceph-mon[29610]: pgmap v708: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-05-09T08:40:58.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:40:58 smithi188 ceph-mon[34645]: pgmap v708: 97 pgs: 97 active+clean; 246 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-05-09T08:40:58.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 687777 7.78 MB/sec execute 434 sec latency 38.521 ms 2024-05-09T08:40:59.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 689189 7.78 MB/sec execute 435 sec latency 31.411 ms 2024-05-09T08:41:00.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:00 smithi160 ceph-mon[34070]: pgmap v709: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:00.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:00 smithi133 ceph-mon[29610]: pgmap v709: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:00.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:00 smithi188 ceph-mon[34645]: pgmap v709: 97 pgs: 97 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:00.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 690607 7.78 MB/sec execute 436 sec latency 13.847 ms 2024-05-09T08:41:01.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 692311 7.78 MB/sec execute 437 sec latency 25.404 ms 2024-05-09T08:41:02.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:02 smithi160 ceph-mon[34070]: pgmap v710: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-09T08:41:02.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:02 smithi133 ceph-mon[29610]: pgmap v710: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-09T08:41:02.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:02 smithi188 ceph-mon[34645]: pgmap v710: 97 pgs: 97 active+clean; 249 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-05-09T08:41:02.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 693775 7.78 MB/sec execute 438 sec latency 10.424 ms 2024-05-09T08:41:03.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 695099 7.78 MB/sec execute 439 sec latency 71.565 ms 2024-05-09T08:41:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:04 smithi160 ceph-mon[34070]: pgmap v711: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:04.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:04 smithi133 ceph-mon[29610]: pgmap v711: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:04.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:04 smithi188 ceph-mon[34645]: pgmap v711: 97 pgs: 97 active+clean; 250 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:04.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 696762 7.78 MB/sec execute 440 sec latency 14.782 ms 2024-05-09T08:41:05.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 698499 7.78 MB/sec execute 441 sec latency 24.410 ms 2024-05-09T08:41:06.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:06 smithi160 ceph-mon[34070]: pgmap v712: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-09T08:41:06.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:06 smithi133 ceph-mon[29610]: pgmap v712: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-09T08:41:06.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:06 smithi188 ceph-mon[34645]: pgmap v712: 97 pgs: 97 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-05-09T08:41:06.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 700053 7.79 MB/sec execute 442 sec latency 20.889 ms 2024-05-09T08:41:07.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 701919 7.79 MB/sec execute 443 sec latency 11.992 ms 2024-05-09T08:41:08.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:08 smithi160 ceph-mon[34070]: pgmap v713: 97 pgs: 97 active+clean; 253 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:41:08.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:08 smithi133 ceph-mon[29610]: pgmap v713: 97 pgs: 97 active+clean; 253 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:41:08.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:08 smithi188 ceph-mon[34645]: pgmap v713: 97 pgs: 97 active+clean; 253 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:41:08.973 INFO:tasks.workunit.client.0.smithi133.stdout: 1 703200 7.80 MB/sec execute 444 sec latency 21.075 ms 2024-05-09T08:41:09.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 704266 7.79 MB/sec execute 445 sec latency 27.012 ms 2024-05-09T08:41:10.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:10 smithi160 ceph-mon[34070]: pgmap v714: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-05-09T08:41:10.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:10 smithi133 ceph-mon[29610]: pgmap v714: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-05-09T08:41:10.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:10 smithi188 ceph-mon[34645]: pgmap v714: 97 pgs: 97 active+clean; 252 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-05-09T08:41:10.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 705721 7.78 MB/sec execute 446 sec latency 26.518 ms 2024-05-09T08:41:11.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 707178 7.79 MB/sec execute 447 sec latency 24.812 ms 2024-05-09T08:41:12.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:12 smithi160 ceph-mon[34070]: pgmap v715: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-09T08:41:12.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:12 smithi133 ceph-mon[29610]: pgmap v715: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-09T08:41:12.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:12 smithi188 ceph-mon[34645]: pgmap v715: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-09T08:41:12.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 708974 7.79 MB/sec execute 448 sec latency 29.869 ms 2024-05-09T08:41:13.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 710225 7.79 MB/sec execute 449 sec latency 48.799 ms 2024-05-09T08:41:14.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:14 smithi160 ceph-mon[34070]: pgmap v716: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-05-09T08:41:14.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:14 smithi133 ceph-mon[29610]: pgmap v716: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-05-09T08:41:14.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:14 smithi188 ceph-mon[34645]: pgmap v716: 97 pgs: 97 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-05-09T08:41:14.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 711296 7.78 MB/sec execute 450 sec latency 52.655 ms 2024-05-09T08:41:15.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 712782 7.78 MB/sec execute 451 sec latency 35.555 ms 2024-05-09T08:41:16.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:16 smithi160 ceph-mon[34070]: pgmap v717: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-05-09T08:41:16.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:16 smithi133 ceph-mon[29610]: pgmap v717: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-05-09T08:41:16.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:16 smithi188 ceph-mon[34645]: pgmap v717: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-05-09T08:41:16.974 INFO:tasks.workunit.client.0.smithi133.stdout: 1 714139 7.79 MB/sec execute 452 sec latency 23.428 ms 2024-05-09T08:41:17.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 716057 7.79 MB/sec execute 453 sec latency 11.409 ms 2024-05-09T08:41:18.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:18 smithi160 ceph-mon[34070]: pgmap v718: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:41:18.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:18 smithi133 ceph-mon[29610]: pgmap v718: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:41:18.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:18 smithi188 ceph-mon[34645]: pgmap v718: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.7 MiB/s wr, 169 op/s 2024-05-09T08:41:18.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 717432 7.79 MB/sec execute 454 sec latency 22.994 ms 2024-05-09T08:41:19.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 718593 7.78 MB/sec execute 455 sec latency 32.176 ms 2024-05-09T08:41:20.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:20 smithi160 ceph-mon[34070]: pgmap v719: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-09T08:41:20.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:20 smithi133 ceph-mon[29610]: pgmap v719: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-09T08:41:20.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:20 smithi188 ceph-mon[34645]: pgmap v719: 97 pgs: 97 active+clean; 256 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-05-09T08:41:20.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 720486 7.79 MB/sec execute 456 sec latency 24.379 ms 2024-05-09T08:41:21.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 722040 7.79 MB/sec execute 457 sec latency 9.778 ms 2024-05-09T08:41:22.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:22 smithi160 ceph-mon[34070]: pgmap v720: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-09T08:41:22.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:22 smithi133 ceph-mon[29610]: pgmap v720: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-09T08:41:22.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:22 smithi188 ceph-mon[34645]: pgmap v720: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-09T08:41:22.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 723784 7.80 MB/sec execute 458 sec latency 23.532 ms 2024-05-09T08:41:23.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 725497 7.80 MB/sec execute 459 sec latency 14.782 ms 2024-05-09T08:41:24.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:24 smithi160 ceph-mon[34070]: pgmap v721: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:41:24.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:24 smithi133 ceph-mon[29610]: pgmap v721: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:41:24.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:24 smithi188 ceph-mon[34645]: pgmap v721: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-09T08:41:24.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 726987 7.79 MB/sec execute 460 sec latency 50.746 ms 2024-05-09T08:41:25.975 INFO:tasks.workunit.client.0.smithi133.stdout: 1 728544 7.80 MB/sec execute 461 sec latency 21.112 ms 2024-05-09T08:41:26.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:26 smithi160 ceph-mon[34070]: pgmap v722: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-05-09T08:41:26.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:26 smithi133 ceph-mon[29610]: pgmap v722: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-05-09T08:41:26.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:26 smithi188 ceph-mon[34645]: pgmap v722: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-05-09T08:41:26.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 730388 7.80 MB/sec execute 462 sec latency 24.803 ms 2024-05-09T08:41:27.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 731906 7.81 MB/sec execute 463 sec latency 18.483 ms 2024-05-09T08:41:28.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:28 smithi160 ceph-mon[34070]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:41:28.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:28 smithi133 ceph-mon[29610]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:41:28.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:28 smithi188 ceph-mon[34645]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-05-09T08:41:28.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 733856 7.81 MB/sec execute 464 sec latency 25.403 ms 2024-05-09T08:41:29.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 735177 7.81 MB/sec execute 465 sec latency 31.105 ms 2024-05-09T08:41:30.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:30 smithi160 ceph-mon[34070]: pgmap v724: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-05-09T08:41:30.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:30 smithi133 ceph-mon[29610]: pgmap v724: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-05-09T08:41:30.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:30 smithi188 ceph-mon[34645]: pgmap v724: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-05-09T08:41:30.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 736705 7.81 MB/sec execute 466 sec latency 11.565 ms 2024-05-09T08:41:31.976 INFO:tasks.workunit.client.0.smithi133.stdout: 1 738550 7.82 MB/sec execute 467 sec latency 31.361 ms 2024-05-09T08:41:32.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:32 smithi160 ceph-mon[34070]: pgmap v725: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-05-09T08:41:32.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:32 smithi133 ceph-mon[29610]: pgmap v725: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-05-09T08:41:32.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:32 smithi188 ceph-mon[34645]: pgmap v725: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 204 op/s 2024-05-09T08:41:32.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 739845 7.81 MB/sec execute 468 sec latency 10.069 ms 2024-05-09T08:41:33.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 741517 7.82 MB/sec execute 469 sec latency 24.760 ms 2024-05-09T08:41:34.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:34 smithi160 ceph-mon[34070]: pgmap v726: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:41:34.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:34 smithi133 ceph-mon[29610]: pgmap v726: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:41:34.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:34 smithi188 ceph-mon[34645]: pgmap v726: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:41:34.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 742850 7.82 MB/sec execute 470 sec latency 20.959 ms 2024-05-09T08:41:35.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 744524 7.81 MB/sec execute 471 sec latency 34.017 ms 2024-05-09T08:41:36.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:36 smithi160 ceph-mon[34070]: pgmap v727: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-05-09T08:41:36.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:36 smithi133 ceph-mon[29610]: pgmap v727: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-05-09T08:41:36.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:36 smithi188 ceph-mon[34645]: pgmap v727: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 211 op/s 2024-05-09T08:41:36.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 745812 7.82 MB/sec execute 472 sec latency 22.361 ms 2024-05-09T08:41:37.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 746745 7.81 MB/sec execute 473 sec latency 16.322 ms 2024-05-09T08:41:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:38 smithi160 ceph-mon[34070]: pgmap v728: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:41:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:38 smithi133 ceph-mon[29610]: pgmap v728: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:41:38.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:38 smithi188 ceph-mon[34645]: pgmap v728: 97 pgs: 97 active+clean; 266 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-05-09T08:41:38.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 748174 7.81 MB/sec execute 474 sec latency 26.288 ms 2024-05-09T08:41:39.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 749448 7.81 MB/sec execute 475 sec latency 33.793 ms 2024-05-09T08:41:40.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:40 smithi160 ceph-mon[34070]: pgmap v729: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:41:40.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:40 smithi133 ceph-mon[29610]: pgmap v729: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:41:40.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:40 smithi188 ceph-mon[34645]: pgmap v729: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:41:40.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 750288 7.80 MB/sec execute 476 sec latency 25.119 ms 2024-05-09T08:41:41.977 INFO:tasks.workunit.client.0.smithi133.stdout: 1 751772 7.80 MB/sec execute 477 sec latency 31.340 ms 2024-05-09T08:41:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:42 smithi160 ceph-mon[34070]: pgmap v730: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:41:42.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:42 smithi133 ceph-mon[29610]: pgmap v730: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:41:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:42 smithi188 ceph-mon[34645]: pgmap v730: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-09T08:41:42.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 753082 7.80 MB/sec execute 478 sec latency 23.571 ms 2024-05-09T08:41:43.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 754203 7.80 MB/sec execute 479 sec latency 15.439 ms 2024-05-09T08:41:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:44 smithi160 ceph-mon[34070]: pgmap v731: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:41:44.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:44 smithi133 ceph-mon[29610]: pgmap v731: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:41:44.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:44 smithi188 ceph-mon[34645]: pgmap v731: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:41:44.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 755373 7.79 MB/sec execute 480 sec latency 79.399 ms 2024-05-09T08:41:45.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 756756 7.80 MB/sec execute 481 sec latency 19.908 ms 2024-05-09T08:41:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:46 smithi160 ceph-mon[34070]: pgmap v732: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-05-09T08:41:46.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:46 smithi133 ceph-mon[29610]: pgmap v732: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-05-09T08:41:46.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:46 smithi188 ceph-mon[34645]: pgmap v732: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-05-09T08:41:46.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 758708 7.79 MB/sec execute 482 sec latency 25.438 ms 2024-05-09T08:41:47.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 760202 7.80 MB/sec execute 483 sec latency 27.477 ms 2024-05-09T08:41:48.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:48 smithi160 ceph-mon[34070]: pgmap v733: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-09T08:41:48.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:48 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:41:48.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:48 smithi133 ceph-mon[29610]: pgmap v733: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-09T08:41:48.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:48 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:41:48.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:48 smithi188 ceph-mon[34645]: pgmap v733: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.9 MiB/s wr, 138 op/s 2024-05-09T08:41:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:48 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:41:48.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 762162 7.80 MB/sec execute 484 sec latency 13.254 ms 2024-05-09T08:41:49.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:41:49.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:41:49.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:41:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:41:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:41:49.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 763527 7.81 MB/sec execute 485 sec latency 42.070 ms 2024-05-09T08:41:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:50 smithi160 ceph-mon[34070]: pgmap v734: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:50.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:50 smithi133 ceph-mon[29610]: pgmap v734: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:50.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:50 smithi188 ceph-mon[34645]: pgmap v734: 97 pgs: 97 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 2.0 MiB/s wr, 145 op/s 2024-05-09T08:41:50.978 INFO:tasks.workunit.client.0.smithi133.stdout: 1 764844 7.80 MB/sec execute 486 sec latency 13.892 ms 2024-05-09T08:41:51.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 766418 7.80 MB/sec execute 487 sec latency 25.880 ms 2024-05-09T08:41:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:52 smithi160 ceph-mon[34070]: pgmap v735: 97 pgs: 97 active+clean; 271 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:41:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:52 smithi133 ceph-mon[29610]: pgmap v735: 97 pgs: 97 active+clean; 271 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:41:52.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:52 smithi188 ceph-mon[34645]: pgmap v735: 97 pgs: 97 active+clean; 271 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-09T08:41:52.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 767766 7.80 MB/sec execute 488 sec latency 15.121 ms 2024-05-09T08:41:53.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 769516 7.80 MB/sec execute 489 sec latency 23.455 ms 2024-05-09T08:41:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:54 smithi160 ceph-mon[34070]: pgmap v736: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-05-09T08:41:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:54 smithi133 ceph-mon[29610]: pgmap v736: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-05-09T08:41:54.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:54 smithi188 ceph-mon[34645]: pgmap v736: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.7 MiB/s wr, 151 op/s 2024-05-09T08:41:54.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 770949 7.81 MB/sec execute 490 sec latency 21.432 ms 2024-05-09T08:41:55.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 772825 7.81 MB/sec execute 491 sec latency 37.117 ms 2024-05-09T08:41:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:56 smithi160 ceph-mon[34070]: pgmap v737: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:56 smithi133 ceph-mon[29610]: pgmap v737: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:56 smithi188 ceph-mon[34645]: pgmap v737: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 198 op/s 2024-05-09T08:41:56.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 774034 7.81 MB/sec execute 492 sec latency 24.387 ms 2024-05-09T08:41:57.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 775184 7.80 MB/sec execute 493 sec latency 11.162 ms 2024-05-09T08:41:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:41:58 smithi160 ceph-mon[34070]: pgmap v738: 97 pgs: 97 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-05-09T08:41:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:41:58 smithi133 ceph-mon[29610]: pgmap v738: 97 pgs: 97 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-05-09T08:41:58.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:41:58 smithi188 ceph-mon[34645]: pgmap v738: 97 pgs: 97 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 159 op/s 2024-05-09T08:41:58.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 776639 7.80 MB/sec execute 494 sec latency 25.245 ms 2024-05-09T08:41:59.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 778013 7.81 MB/sec execute 495 sec latency 23.679 ms 2024-05-09T08:42:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:00 smithi160 ceph-mon[34070]: pgmap v739: 97 pgs: 97 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:00 smithi133 ceph-mon[29610]: pgmap v739: 97 pgs: 97 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:00.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:00 smithi188 ceph-mon[34645]: pgmap v739: 97 pgs: 97 active+clean; 275 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:00.979 INFO:tasks.workunit.client.0.smithi133.stdout: 1 779464 7.80 MB/sec execute 496 sec latency 58.077 ms 2024-05-09T08:42:01.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 780927 7.80 MB/sec execute 497 sec latency 25.695 ms 2024-05-09T08:42:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:02 smithi160 ceph-mon[34070]: pgmap v740: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:02 smithi133 ceph-mon[29610]: pgmap v740: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:02.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:02 smithi188 ceph-mon[34645]: pgmap v740: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:02.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 782051 7.80 MB/sec execute 498 sec latency 13.791 ms 2024-05-09T08:42:03.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 783607 7.80 MB/sec execute 499 sec latency 26.517 ms 2024-05-09T08:42:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:04 smithi160 ceph-mon[34070]: pgmap v741: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-09T08:42:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:04 smithi133 ceph-mon[29610]: pgmap v741: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-09T08:42:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:04 smithi188 ceph-mon[34645]: pgmap v741: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-05-09T08:42:04.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 785008 7.80 MB/sec execute 500 sec latency 27.995 ms 2024-05-09T08:42:05.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 786240 7.80 MB/sec execute 501 sec latency 56.046 ms 2024-05-09T08:42:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:06 smithi160 ceph-mon[34070]: pgmap v742: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-05-09T08:42:06.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:06 smithi133 ceph-mon[29610]: pgmap v742: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-05-09T08:42:06.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:06 smithi188 ceph-mon[34645]: pgmap v742: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-05-09T08:42:06.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 787596 7.80 MB/sec execute 502 sec latency 30.545 ms 2024-05-09T08:42:07.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 788769 7.80 MB/sec execute 503 sec latency 12.884 ms 2024-05-09T08:42:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:08 smithi160 ceph-mon[34070]: pgmap v743: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-09T08:42:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:08 smithi133 ceph-mon[29610]: pgmap v743: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-09T08:42:08.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:08 smithi188 ceph-mon[34645]: pgmap v743: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-09T08:42:08.980 INFO:tasks.workunit.client.0.smithi133.stdout: 1 790568 7.79 MB/sec execute 504 sec latency 31.095 ms 2024-05-09T08:42:09.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 791852 7.80 MB/sec execute 505 sec latency 20.598 ms 2024-05-09T08:42:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:10 smithi160 ceph-mon[34070]: pgmap v744: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-05-09T08:42:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:10 smithi133 ceph-mon[29610]: pgmap v744: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-05-09T08:42:10.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:10 smithi188 ceph-mon[34645]: pgmap v744: 97 pgs: 97 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-05-09T08:42:10.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 792656 7.79 MB/sec execute 506 sec latency 47.998 ms 2024-05-09T08:42:11.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 794227 7.79 MB/sec execute 507 sec latency 31.253 ms 2024-05-09T08:42:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:12 smithi160 ceph-mon[34070]: pgmap v745: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-05-09T08:42:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:12 smithi133 ceph-mon[29610]: pgmap v745: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-05-09T08:42:12.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:12 smithi188 ceph-mon[34645]: pgmap v745: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-05-09T08:42:12.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 795657 7.80 MB/sec execute 508 sec latency 22.042 ms 2024-05-09T08:42:13.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 796990 7.79 MB/sec execute 509 sec latency 17.745 ms 2024-05-09T08:42:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:14 smithi160 ceph-mon[34070]: pgmap v746: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:42:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:14 smithi133 ceph-mon[29610]: pgmap v746: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:42:14.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:14 smithi188 ceph-mon[34645]: pgmap v746: 97 pgs: 97 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-05-09T08:42:14.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 798330 7.79 MB/sec execute 510 sec latency 36.847 ms 2024-05-09T08:42:15.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 799541 7.79 MB/sec execute 511 sec latency 56.086 ms 2024-05-09T08:42:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:16 smithi160 ceph-mon[34070]: pgmap v747: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:42:16.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:16 smithi133 ceph-mon[29610]: pgmap v747: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:42:16.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:16 smithi188 ceph-mon[34645]: pgmap v747: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:42:16.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 801263 7.79 MB/sec execute 512 sec latency 29.353 ms 2024-05-09T08:42:17.981 INFO:tasks.workunit.client.0.smithi133.stdout: 1 802632 7.79 MB/sec execute 513 sec latency 24.931 ms 2024-05-09T08:42:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:18 smithi160 ceph-mon[34070]: pgmap v748: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:42:18.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:18 smithi133 ceph-mon[29610]: pgmap v748: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:42:18.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:18 smithi188 ceph-mon[34645]: pgmap v748: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:42:18.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 803836 7.79 MB/sec execute 514 sec latency 15.847 ms 2024-05-09T08:42:19.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 805018 7.78 MB/sec execute 515 sec latency 31.594 ms 2024-05-09T08:42:20.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:20 smithi160 ceph-mon[34070]: pgmap v749: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-05-09T08:42:20.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:20 smithi133 ceph-mon[29610]: pgmap v749: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-05-09T08:42:20.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:20 smithi188 ceph-mon[34645]: pgmap v749: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-05-09T08:42:20.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 806388 7.79 MB/sec execute 516 sec latency 29.598 ms 2024-05-09T08:42:21.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 807934 7.78 MB/sec execute 517 sec latency 21.579 ms 2024-05-09T08:42:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:22 smithi160 ceph-mon[34070]: pgmap v750: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:42:22.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:22 smithi133 ceph-mon[29610]: pgmap v750: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:42:22.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:22 smithi188 ceph-mon[34645]: pgmap v750: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:42:22.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 809120 7.78 MB/sec execute 518 sec latency 39.772 ms 2024-05-09T08:42:23.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 810100 7.78 MB/sec execute 519 sec latency 18.362 ms 2024-05-09T08:42:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:24 smithi160 ceph-mon[34070]: pgmap v751: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:42:24.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:24 smithi133 ceph-mon[29610]: pgmap v751: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:42:24.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:24 smithi188 ceph-mon[34645]: pgmap v751: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:42:24.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 811799 7.78 MB/sec execute 520 sec latency 65.981 ms 2024-05-09T08:42:25.982 INFO:tasks.workunit.client.0.smithi133.stdout: 1 812812 7.78 MB/sec execute 521 sec latency 44.994 ms 2024-05-09T08:42:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:26 smithi160 ceph-mon[34070]: pgmap v752: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-05-09T08:42:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:26 smithi133 ceph-mon[29610]: pgmap v752: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-05-09T08:42:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:26 smithi188 ceph-mon[34645]: pgmap v752: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-05-09T08:42:26.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 813968 7.77 MB/sec execute 522 sec latency 18.078 ms 2024-05-09T08:42:27.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 815606 7.77 MB/sec execute 523 sec latency 22.663 ms 2024-05-09T08:42:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:28 smithi160 ceph-mon[34070]: pgmap v753: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:42:28.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:28 smithi133 ceph-mon[29610]: pgmap v753: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:42:28.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:28 smithi188 ceph-mon[34645]: pgmap v753: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 1.9 MiB/s wr, 157 op/s 2024-05-09T08:42:28.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 817136 7.78 MB/sec execute 524 sec latency 21.228 ms 2024-05-09T08:42:29.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 818995 7.77 MB/sec execute 525 sec latency 29.874 ms 2024-05-09T08:42:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:30 smithi160 ceph-mon[34070]: pgmap v754: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-05-09T08:42:30.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:30 smithi133 ceph-mon[29610]: pgmap v754: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-05-09T08:42:30.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:30 smithi188 ceph-mon[34645]: pgmap v754: 97 pgs: 97 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-05-09T08:42:30.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 820352 7.78 MB/sec execute 526 sec latency 22.832 ms 2024-05-09T08:42:31.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 821758 7.78 MB/sec execute 527 sec latency 22.620 ms 2024-05-09T08:42:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:32 smithi160 ceph-mon[34070]: pgmap v755: 97 pgs: 97 active+clean; 288 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-09T08:42:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:32 smithi133 ceph-mon[29610]: pgmap v755: 97 pgs: 97 active+clean; 288 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-09T08:42:32.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:32 smithi188 ceph-mon[34645]: pgmap v755: 97 pgs: 97 active+clean; 288 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 2.0 MiB/s wr, 160 op/s 2024-05-09T08:42:32.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 823345 7.78 MB/sec execute 528 sec latency 24.330 ms 2024-05-09T08:42:33.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 824607 7.78 MB/sec execute 529 sec latency 16.398 ms 2024-05-09T08:42:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:34 smithi160 ceph-mon[34070]: pgmap v756: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-05-09T08:42:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:34 smithi133 ceph-mon[29610]: pgmap v756: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-05-09T08:42:34.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:34 smithi188 ceph-mon[34645]: pgmap v756: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-05-09T08:42:34.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 826254 7.78 MB/sec execute 530 sec latency 28.298 ms 2024-05-09T08:42:35.983 INFO:tasks.workunit.client.0.smithi133.stdout: 1 827765 7.78 MB/sec execute 531 sec latency 29.000 ms 2024-05-09T08:42:36.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:36 smithi160 ceph-mon[34070]: pgmap v757: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-05-09T08:42:36.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:36 smithi133 ceph-mon[29610]: pgmap v757: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-05-09T08:42:36.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:36 smithi188 ceph-mon[34645]: pgmap v757: 97 pgs: 97 active+clean; 290 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 171 op/s 2024-05-09T08:42:36.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 829428 7.78 MB/sec execute 532 sec latency 76.036 ms 2024-05-09T08:42:37.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 830538 7.78 MB/sec execute 533 sec latency 28.477 ms 2024-05-09T08:42:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:38 smithi160 ceph-mon[34070]: pgmap v758: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-05-09T08:42:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:38 smithi133 ceph-mon[29610]: pgmap v758: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-05-09T08:42:38.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:38 smithi188 ceph-mon[34645]: pgmap v758: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-05-09T08:42:38.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 831650 7.77 MB/sec execute 534 sec latency 19.087 ms 2024-05-09T08:42:39.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 833217 7.77 MB/sec execute 535 sec latency 27.629 ms 2024-05-09T08:42:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:40 smithi160 ceph-mon[34070]: pgmap v759: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:42:40.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:40 smithi133 ceph-mon[29610]: pgmap v759: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:42:40.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:40 smithi188 ceph-mon[34645]: pgmap v759: 97 pgs: 97 active+clean; 294 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-05-09T08:42:40.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 834574 7.78 MB/sec execute 536 sec latency 25.383 ms 2024-05-09T08:42:41.984 INFO:tasks.workunit.client.0.smithi133.stdout: 1 835753 7.78 MB/sec execute 537 sec latency 42.770 ms 2024-05-09T08:42:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:42 smithi160 ceph-mon[34070]: pgmap v760: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:42 smithi133 ceph-mon[29610]: pgmap v760: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:42 smithi188 ceph-mon[34645]: pgmap v760: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-05-09T08:42:42.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 836953 7.77 MB/sec execute 538 sec latency 26.645 ms 2024-05-09T08:42:43.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 838352 7.77 MB/sec execute 539 sec latency 26.228 ms 2024-05-09T08:42:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:44 smithi160 ceph-mon[34070]: pgmap v761: 97 pgs: 97 active+clean; 295 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:44.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:44 smithi133 ceph-mon[29610]: pgmap v761: 97 pgs: 97 active+clean; 295 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:44.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:44 smithi188 ceph-mon[34645]: pgmap v761: 97 pgs: 97 active+clean; 295 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 2.1 MiB/s wr, 180 op/s 2024-05-09T08:42:44.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 839787 7.77 MB/sec execute 540 sec latency 23.343 ms 2024-05-09T08:42:45.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 840919 7.77 MB/sec execute 541 sec latency 30.698 ms 2024-05-09T08:42:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:46 smithi160 ceph-mon[34070]: pgmap v762: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-05-09T08:42:46.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:46 smithi133 ceph-mon[29610]: pgmap v762: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-05-09T08:42:46.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:46 smithi188 ceph-mon[34645]: pgmap v762: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-05-09T08:42:46.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 842063 7.76 MB/sec execute 542 sec latency 40.823 ms 2024-05-09T08:42:47.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 843863 7.76 MB/sec execute 543 sec latency 26.445 ms 2024-05-09T08:42:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:48 smithi160 ceph-mon[34070]: pgmap v763: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-05-09T08:42:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:48 smithi133 ceph-mon[29610]: pgmap v763: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-05-09T08:42:48.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:48 smithi188 ceph-mon[34645]: pgmap v763: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 160 op/s 2024-05-09T08:42:48.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 845229 7.77 MB/sec execute 544 sec latency 23.300 ms 2024-05-09T08:42:49.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:49 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:42:49.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:49 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:42:49.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:49 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:42:49.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 846550 7.77 MB/sec execute 545 sec latency 18.346 ms 2024-05-09T08:42:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:50 smithi160 ceph-mon[34070]: pgmap v764: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-09T08:42:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:50 smithi133 ceph-mon[29610]: pgmap v764: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-09T08:42:50.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:50 smithi188 ceph-mon[34645]: pgmap v764: 97 pgs: 97 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-05-09T08:42:50.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 848369 7.77 MB/sec execute 546 sec latency 27.499 ms 2024-05-09T08:42:51.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:42:51.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:42:51.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:42:51.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:42:51.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:42:51.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:42:51.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:42:51.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:42:51.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:42:51.985 INFO:tasks.workunit.client.0.smithi133.stdout: 1 849376 7.76 MB/sec execute 547 sec latency 35.763 ms 2024-05-09T08:42:52.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 851018 7.76 MB/sec execute 548 sec latency 31.186 ms 2024-05-09T08:42:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:52 smithi160 ceph-mon[34070]: pgmap v765: 97 pgs: 97 active+clean; 298 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-05-09T08:42:53.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:52 smithi133 ceph-mon[29610]: pgmap v765: 97 pgs: 97 active+clean; 298 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-05-09T08:42:53.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:52 smithi188 ceph-mon[34645]: pgmap v765: 97 pgs: 97 active+clean; 298 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-05-09T08:42:53.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 852464 7.77 MB/sec execute 549 sec latency 25.020 ms 2024-05-09T08:42:54.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 853972 7.77 MB/sec execute 550 sec latency 16.739 ms 2024-05-09T08:42:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:54 smithi160 ceph-mon[34070]: pgmap v766: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-09T08:42:55.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:54 smithi133 ceph-mon[29610]: pgmap v766: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-09T08:42:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:54 smithi188 ceph-mon[34645]: pgmap v766: 97 pgs: 97 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-05-09T08:42:55.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 855278 7.77 MB/sec execute 551 sec latency 30.080 ms 2024-05-09T08:42:56.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 856434 7.76 MB/sec execute 552 sec latency 55.554 ms 2024-05-09T08:42:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:56 smithi160 ceph-mon[34070]: pgmap v767: 97 pgs: 97 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-09T08:42:57.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:56 smithi133 ceph-mon[29610]: pgmap v767: 97 pgs: 97 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-09T08:42:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:56 smithi188 ceph-mon[34645]: pgmap v767: 97 pgs: 97 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-05-09T08:42:57.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 858144 7.76 MB/sec execute 553 sec latency 21.128 ms 2024-05-09T08:42:58.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 859747 7.77 MB/sec execute 554 sec latency 22.898 ms 2024-05-09T08:42:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:42:58 smithi160 ceph-mon[34070]: pgmap v768: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:42:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:42:58 smithi133 ceph-mon[29610]: pgmap v768: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:42:59.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:42:58 smithi188 ceph-mon[34645]: pgmap v768: 97 pgs: 97 active+clean; 301 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:42:59.986 INFO:tasks.workunit.client.0.smithi133.stdout: 1 861592 7.77 MB/sec execute 555 sec latency 31.095 ms 2024-05-09T08:43:00.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 862993 7.78 MB/sec execute 556 sec latency 22.347 ms 2024-05-09T08:43:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:00 smithi160 ceph-mon[34070]: pgmap v769: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:43:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:00 smithi133 ceph-mon[29610]: pgmap v769: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:43:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:00 smithi188 ceph-mon[34645]: pgmap v769: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-05-09T08:43:01.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 864353 7.77 MB/sec execute 557 sec latency 57.991 ms 2024-05-09T08:43:02.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 865939 7.77 MB/sec execute 558 sec latency 37.393 ms 2024-05-09T08:43:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:02 smithi160 ceph-mon[34070]: pgmap v770: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-05-09T08:43:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:02 smithi133 ceph-mon[29610]: pgmap v770: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-05-09T08:43:03.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:02 smithi188 ceph-mon[34645]: pgmap v770: 97 pgs: 97 active+clean; 304 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-05-09T08:43:03.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 867343 7.77 MB/sec execute 559 sec latency 16.397 ms 2024-05-09T08:43:04.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 868889 7.77 MB/sec execute 560 sec latency 22.022 ms 2024-05-09T08:43:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:04 smithi160 ceph-mon[34070]: pgmap v771: 97 pgs: 97 active+clean; 305 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:43:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:04 smithi133 ceph-mon[29610]: pgmap v771: 97 pgs: 97 active+clean; 305 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:43:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:04 smithi188 ceph-mon[34645]: pgmap v771: 97 pgs: 97 active+clean; 305 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-05-09T08:43:05.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 870502 7.77 MB/sec execute 561 sec latency 16.920 ms 2024-05-09T08:43:06.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 872255 7.77 MB/sec execute 562 sec latency 28.472 ms 2024-05-09T08:43:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:06 smithi160 ceph-mon[34070]: pgmap v772: 97 pgs: 97 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 196 op/s 2024-05-09T08:43:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:06 smithi133 ceph-mon[29610]: pgmap v772: 97 pgs: 97 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 196 op/s 2024-05-09T08:43:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:06 smithi188 ceph-mon[34645]: pgmap v772: 97 pgs: 97 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 196 op/s 2024-05-09T08:43:07.987 INFO:tasks.workunit.client.0.smithi133.stdout: 1 873670 7.78 MB/sec execute 563 sec latency 28.342 ms 2024-05-09T08:43:08.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 875444 7.78 MB/sec execute 564 sec latency 16.730 ms 2024-05-09T08:43:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:08 smithi160 ceph-mon[34070]: pgmap v773: 97 pgs: 97 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-05-09T08:43:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:08 smithi133 ceph-mon[29610]: pgmap v773: 97 pgs: 97 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-05-09T08:43:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:08 smithi188 ceph-mon[34645]: pgmap v773: 97 pgs: 97 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.9 MiB/s wr, 150 op/s 2024-05-09T08:43:09.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 876935 7.78 MB/sec execute 565 sec latency 26.420 ms 2024-05-09T08:43:10.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 878192 7.78 MB/sec execute 566 sec latency 16.384 ms 2024-05-09T08:43:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:10 smithi160 ceph-mon[34070]: pgmap v774: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-05-09T08:43:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:10 smithi133 ceph-mon[29610]: pgmap v774: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-05-09T08:43:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:10 smithi188 ceph-mon[34645]: pgmap v774: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-05-09T08:43:11.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 879841 7.78 MB/sec execute 567 sec latency 62.207 ms 2024-05-09T08:43:12.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 881208 7.78 MB/sec execute 568 sec latency 19.615 ms 2024-05-09T08:43:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:12 smithi160 ceph-mon[34070]: pgmap v775: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-05-09T08:43:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:12 smithi133 ceph-mon[29610]: pgmap v775: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-05-09T08:43:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:12 smithi188 ceph-mon[34645]: pgmap v775: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 170 op/s 2024-05-09T08:43:13.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 882925 7.78 MB/sec execute 569 sec latency 29.984 ms 2024-05-09T08:43:14.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 884403 7.79 MB/sec execute 570 sec latency 23.607 ms 2024-05-09T08:43:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:14 smithi160 ceph-mon[34070]: pgmap v776: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:43:15.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:14 smithi133 ceph-mon[29610]: pgmap v776: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:43:15.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:14 smithi188 ceph-mon[34645]: pgmap v776: 97 pgs: 97 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 2.0 MiB/s wr, 162 op/s 2024-05-09T08:43:15.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 885909 7.78 MB/sec execute 571 sec latency 19.526 ms 2024-05-09T08:43:16.988 INFO:tasks.workunit.client.0.smithi133.stdout: 1 887009 7.78 MB/sec execute 572 sec latency 58.648 ms 2024-05-09T08:43:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:16 smithi160 ceph-mon[34070]: pgmap v777: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-09T08:43:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:16 smithi133 ceph-mon[29610]: pgmap v777: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-09T08:43:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:16 smithi188 ceph-mon[34645]: pgmap v777: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 210 op/s 2024-05-09T08:43:17.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 888250 7.78 MB/sec execute 573 sec latency 25.896 ms 2024-05-09T08:43:18.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 889953 7.78 MB/sec execute 574 sec latency 33.739 ms 2024-05-09T08:43:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:18 smithi160 ceph-mon[34070]: pgmap v778: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:43:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:18 smithi133 ceph-mon[29610]: pgmap v778: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:43:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:18 smithi188 ceph-mon[34645]: pgmap v778: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-09T08:43:19.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 891107 7.78 MB/sec execute 575 sec latency 28.095 ms 2024-05-09T08:43:20.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 892046 7.77 MB/sec execute 576 sec latency 17.310 ms 2024-05-09T08:43:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:20 smithi160 ceph-mon[34070]: pgmap v779: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:43:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:20 smithi133 ceph-mon[29610]: pgmap v779: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:43:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:20 smithi188 ceph-mon[34645]: pgmap v779: 97 pgs: 97 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-05-09T08:43:21.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 893545 7.77 MB/sec execute 577 sec latency 41.119 ms 2024-05-09T08:43:22.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 894981 7.78 MB/sec execute 578 sec latency 52.191 ms 2024-05-09T08:43:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:22 smithi160 ceph-mon[34070]: pgmap v780: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:43:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:22 smithi133 ceph-mon[29610]: pgmap v780: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:43:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:22 smithi188 ceph-mon[34645]: pgmap v780: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-05-09T08:43:23.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 896188 7.77 MB/sec execute 579 sec latency 20.889 ms 2024-05-09T08:43:24.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 897265 7.76 MB/sec execute 580 sec latency 32.640 ms 2024-05-09T08:43:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:24 smithi160 ceph-mon[34070]: pgmap v781: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-05-09T08:43:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:24 smithi133 ceph-mon[29610]: pgmap v781: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-05-09T08:43:25.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:24 smithi188 ceph-mon[34645]: pgmap v781: 97 pgs: 97 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-05-09T08:43:25.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 898642 7.77 MB/sec execute 581 sec latency 24.776 ms 2024-05-09T08:43:26.989 INFO:tasks.workunit.client.0.smithi133.stdout: 1 900193 7.77 MB/sec execute 582 sec latency 19.873 ms 2024-05-09T08:43:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:26 smithi160 ceph-mon[34070]: pgmap v782: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:43:27.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:26 smithi133 ceph-mon[29610]: pgmap v782: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:43:27.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:26 smithi188 ceph-mon[34645]: pgmap v782: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-05-09T08:43:27.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 901540 7.77 MB/sec execute 583 sec latency 73.132 ms 2024-05-09T08:43:28.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 902623 7.76 MB/sec execute 584 sec latency 19.541 ms 2024-05-09T08:43:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:28 smithi160 ceph-mon[34070]: pgmap v783: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:28 smithi133 ceph-mon[29610]: pgmap v783: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:28 smithi188 ceph-mon[34645]: pgmap v783: 97 pgs: 97 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:29.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 904178 7.76 MB/sec execute 585 sec latency 40.711 ms 2024-05-09T08:43:30.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 905534 7.77 MB/sec execute 586 sec latency 25.930 ms 2024-05-09T08:43:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:30 smithi160 ceph-mon[34070]: pgmap v784: 97 pgs: 97 active+clean; 316 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:43:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:30 smithi133 ceph-mon[29610]: pgmap v784: 97 pgs: 97 active+clean; 316 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:43:31.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:30 smithi188 ceph-mon[34645]: pgmap v784: 97 pgs: 97 active+clean; 316 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-05-09T08:43:31.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 906511 7.76 MB/sec execute 587 sec latency 22.200 ms 2024-05-09T08:43:32.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 907867 7.76 MB/sec execute 588 sec latency 35.871 ms 2024-05-09T08:43:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:32 smithi133 ceph-mon[29610]: pgmap v785: 97 pgs: 97 active+clean; 317 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-05-09T08:43:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:32 smithi188 ceph-mon[34645]: pgmap v785: 97 pgs: 97 active+clean; 317 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-05-09T08:43:33.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:32 smithi160 ceph-mon[34070]: pgmap v785: 97 pgs: 97 active+clean; 317 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-05-09T08:43:33.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 909270 7.76 MB/sec execute 589 sec latency 26.745 ms 2024-05-09T08:43:34.990 INFO:tasks.workunit.client.0.smithi133.stdout: 1 910477 7.76 MB/sec execute 590 sec latency 22.416 ms 2024-05-09T08:43:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:34 smithi133 ceph-mon[29610]: pgmap v786: 97 pgs: 97 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-09T08:43:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:34 smithi188 ceph-mon[34645]: pgmap v786: 97 pgs: 97 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-09T08:43:35.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:34 smithi160 ceph-mon[34070]: pgmap v786: 97 pgs: 97 active+clean; 308 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-05-09T08:43:35.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 911601 7.75 MB/sec execute 591 sec latency 40.547 ms 2024-05-09T08:43:36.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 912916 7.76 MB/sec execute 592 sec latency 11.303 ms 2024-05-09T08:43:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:36 smithi133 ceph-mon[29610]: pgmap v787: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:43:37.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:36 smithi188 ceph-mon[34645]: pgmap v787: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:43:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:36 smithi160 ceph-mon[34070]: pgmap v787: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 2.1 MiB/s wr, 182 op/s 2024-05-09T08:43:37.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 914524 7.75 MB/sec execute 593 sec latency 39.667 ms 2024-05-09T08:43:38.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 915676 7.75 MB/sec execute 594 sec latency 31.681 ms 2024-05-09T08:43:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:38 smithi188 ceph-mon[34645]: pgmap v788: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:39.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:38 smithi160 ceph-mon[34070]: pgmap v788: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:39.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:38 smithi133 ceph-mon[29610]: pgmap v788: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-05-09T08:43:39.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 916688 7.75 MB/sec execute 595 sec latency 52.617 ms 2024-05-09T08:43:40.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 917658 7.77 MB/sec execute 596 sec latency 97.981 ms 2024-05-09T08:43:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:40 smithi160 ceph-mon[34070]: pgmap v789: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-09T08:43:41.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:40 smithi133 ceph-mon[29610]: pgmap v789: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-09T08:43:41.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:40 smithi188 ceph-mon[34645]: pgmap v789: 97 pgs: 97 active+clean; 294 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-05-09T08:43:41.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 919154 7.77 MB/sec execute 597 sec latency 28.751 ms 2024-05-09T08:43:42.333 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:41 smithi188 ceph-mon[34645]: pgmap v790: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:43:42.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:41 smithi160 ceph-mon[34070]: pgmap v790: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:43:42.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:41 smithi133 ceph-mon[29610]: pgmap v790: 97 pgs: 97 active+clean; 297 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 2.1 MiB/s wr, 173 op/s 2024-05-09T08:43:42.991 INFO:tasks.workunit.client.0.smithi133.stdout: 1 920516 7.78 MB/sec execute 598 sec latency 68.031 ms 2024-05-09T08:43:43.992 INFO:tasks.workunit.client.0.smithi133.stdout: 1 921766 7.77 MB/sec execute 599 sec latency 21.146 ms 2024-05-09T08:43:44.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:44 smithi133 ceph-mon[29610]: pgmap v791: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:43:44.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:44 smithi188 ceph-mon[34645]: pgmap v791: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:43:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:44 smithi160 ceph-mon[34070]: pgmap v791: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 1.9 MiB/s wr, 155 op/s 2024-05-09T08:43:44.992 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 600 sec 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: 0 cleanup 600 sec 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: Operation Count AvgLat MaxLat 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: ---------------------------------------- 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: NTCreateX 148411 1.381 79.445 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: Close 109060 0.010 0.668 2024-05-09T08:43:45.228 INFO:tasks.workunit.client.0.smithi133.stdout: Rename 6287 7.307 79.385 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Unlink 29964 2.453 199.368 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Deltree 4 113.755 144.508 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Mkdir 2 0.080 0.093 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Qpathinfo 134581 1.069 78.726 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Qfileinfo 23602 0.005 0.268 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Qfsinfo 24657 0.716 43.664 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Sfileinfo 12108 0.336 71.049 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Find 52016 0.336 58.976 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: WriteX 74082 0.081 13.127 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: ReadX 232663 0.084 696.635 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: LockX 484 0.908 22.527 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: UnlockX 484 0.755 6.200 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: Flush 10411 5.472 94.126 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:43:45.229 INFO:tasks.workunit.client.0.smithi133.stdout:Throughput 7.77224 MB/sec 1 clients 1 procs max_latency=696.666 ms 2024-05-09T08:43:45.230 INFO:tasks.workunit.client.0.smithi133.stderr:+ dbench 10 2024-05-09T08:43:45.232 INFO:tasks.workunit.client.0.smithi133.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-09T08:43:45.232 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:43:45.232 INFO:tasks.workunit.client.0.smithi133.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-09T08:43:45.236 INFO:tasks.workunit.client.0.smithi133.stdout:9 of 10 processes prepared for launch 0 sec 2024-05-09T08:43:45.337 INFO:tasks.workunit.client.0.smithi133.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-09T08:43:45.337 INFO:tasks.workunit.client.0.smithi133.stdout:releasing clients 2024-05-09T08:43:46.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 445 156.70 MB/sec warmup 1 sec latency 36.397 ms 2024-05-09T08:43:46.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:46 smithi188 ceph-mon[34645]: pgmap v792: 97 pgs: 97 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:43:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:46 smithi160 ceph-mon[34070]: pgmap v792: 97 pgs: 97 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:43:46.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:46 smithi133 ceph-mon[29610]: pgmap v792: 97 pgs: 97 active+clean; 290 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-05-09T08:43:47.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 972 109.60 MB/sec warmup 2 sec latency 63.122 ms 2024-05-09T08:43:48.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 1565 87.06 MB/sec warmup 3 sec latency 57.211 ms 2024-05-09T08:43:48.463 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:48 smithi160 ceph-mon[34070]: pgmap v793: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.3 MiB/s wr, 165 op/s 2024-05-09T08:43:48.478 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:48 smithi133 ceph-mon[29610]: pgmap v793: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.3 MiB/s wr, 165 op/s 2024-05-09T08:43:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:48 smithi188 ceph-mon[34645]: pgmap v793: 97 pgs: 97 active+clean; 288 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.3 MiB/s wr, 165 op/s 2024-05-09T08:43:49.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 2374 69.69 MB/sec warmup 4 sec latency 109.658 ms 2024-05-09T08:43:50.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 3298 69.87 MB/sec warmup 5 sec latency 78.413 ms 2024-05-09T08:43:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:50 smithi160 ceph-mon[34070]: pgmap v794: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-05-09T08:43:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:50 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:43:50.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:50 smithi133 ceph-mon[29610]: pgmap v794: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-05-09T08:43:50.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:50 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:43:50.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:50 smithi188 ceph-mon[34645]: pgmap v794: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-05-09T08:43:50.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:50 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:43:51.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 4060 67.40 MB/sec warmup 6 sec latency 50.249 ms 2024-05-09T08:43:51.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:51 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:43:51.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:51 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:43:51.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:51 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:43:52.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 4475 60.24 MB/sec warmup 7 sec latency 70.946 ms 2024-05-09T08:43:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:52 smithi160 ceph-mon[34070]: pgmap v795: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-05-09T08:43:52.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:52 smithi133 ceph-mon[29610]: pgmap v795: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-05-09T08:43:52.681 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:52 smithi188 ceph-mon[34645]: pgmap v795: 97 pgs: 97 active+clean; 296 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 3.0 MiB/s wr, 225 op/s 2024-05-09T08:43:53.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 4994 54.68 MB/sec warmup 8 sec latency 140.693 ms 2024-05-09T08:43:53.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:53 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:43:53.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:53 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:43:53.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:53 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:43:53.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:53 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:43:53.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:53 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:43:53.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:53 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:43:53.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:53 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:43:53.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:53 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:43:53.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:53 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:43:54.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 5667 53.44 MB/sec warmup 9 sec latency 60.958 ms 2024-05-09T08:43:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:54 smithi160 ceph-mon[34070]: pgmap v796: 97 pgs: 97 active+clean; 298 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:43:54.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:54 smithi133 ceph-mon[29610]: pgmap v796: 97 pgs: 97 active+clean; 298 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:43:54.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:54 smithi188 ceph-mon[34645]: pgmap v796: 97 pgs: 97 active+clean; 298 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 2.8 MiB/s wr, 196 op/s 2024-05-09T08:43:55.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 6545 51.62 MB/sec warmup 10 sec latency 92.233 ms 2024-05-09T08:43:56.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 7467 52.03 MB/sec warmup 11 sec latency 30.917 ms 2024-05-09T08:43:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:56 smithi160 ceph-mon[34070]: pgmap v797: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-09T08:43:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:56 smithi133 ceph-mon[29610]: pgmap v797: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-09T08:43:56.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:56 smithi188 ceph-mon[34645]: pgmap v797: 97 pgs: 97 active+clean; 309 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 322 op/s 2024-05-09T08:43:57.337 INFO:tasks.workunit.client.0.smithi133.stdout: 10 8276 52.29 MB/sec warmup 12 sec latency 48.594 ms 2024-05-09T08:43:58.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 8731 50.26 MB/sec warmup 13 sec latency 138.652 ms 2024-05-09T08:43:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:43:58 smithi160 ceph-mon[34070]: pgmap v798: 97 pgs: 97 active+clean; 313 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-05-09T08:43:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:43:58 smithi133 ceph-mon[29610]: pgmap v798: 97 pgs: 97 active+clean; 313 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-05-09T08:43:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:43:58 smithi188 ceph-mon[34645]: pgmap v798: 97 pgs: 97 active+clean; 313 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 346 op/s 2024-05-09T08:43:59.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 9238 48.12 MB/sec warmup 14 sec latency 151.251 ms 2024-05-09T08:44:00.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 9922 47.14 MB/sec warmup 15 sec latency 83.023 ms 2024-05-09T08:44:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:00 smithi160 ceph-mon[34070]: pgmap v799: 97 pgs: 97 active+clean; 320 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 388 op/s 2024-05-09T08:44:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:00 smithi133 ceph-mon[29610]: pgmap v799: 97 pgs: 97 active+clean; 320 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 388 op/s 2024-05-09T08:44:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:00 smithi188 ceph-mon[34645]: pgmap v799: 97 pgs: 97 active+clean; 320 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 388 op/s 2024-05-09T08:44:01.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 10775 46.29 MB/sec warmup 16 sec latency 52.673 ms 2024-05-09T08:44:02.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 11740 47.35 MB/sec warmup 17 sec latency 47.659 ms 2024-05-09T08:44:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:02 smithi160 ceph-mon[34070]: pgmap v800: 97 pgs: 97 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-05-09T08:44:02.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:02 smithi133 ceph-mon[29610]: pgmap v800: 97 pgs: 97 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-05-09T08:44:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:02 smithi188 ceph-mon[34645]: pgmap v800: 97 pgs: 97 active+clean; 327 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 449 op/s 2024-05-09T08:44:03.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 12309 46.79 MB/sec warmup 18 sec latency 268.312 ms 2024-05-09T08:44:04.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 12775 45.63 MB/sec warmup 19 sec latency 95.118 ms 2024-05-09T08:44:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:04 smithi160 ceph-mon[34070]: pgmap v801: 97 pgs: 97 active+clean; 329 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 393 op/s 2024-05-09T08:44:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:04 smithi133 ceph-mon[29610]: pgmap v801: 97 pgs: 97 active+clean; 329 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 393 op/s 2024-05-09T08:44:04.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:04 smithi188 ceph-mon[34645]: pgmap v801: 97 pgs: 97 active+clean; 329 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 393 op/s 2024-05-09T08:44:05.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 13411 44.93 MB/sec warmup 20 sec latency 106.253 ms 2024-05-09T08:44:06.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 14040 44.47 MB/sec warmup 21 sec latency 73.817 ms 2024-05-09T08:44:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:06 smithi160 ceph-mon[34070]: pgmap v802: 97 pgs: 97 active+clean; 352 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 549 op/s 2024-05-09T08:44:06.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:06 smithi133 ceph-mon[29610]: pgmap v802: 97 pgs: 97 active+clean; 352 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 549 op/s 2024-05-09T08:44:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:06 smithi188 ceph-mon[34645]: pgmap v802: 97 pgs: 97 active+clean; 352 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 549 op/s 2024-05-09T08:44:07.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 14754 43.56 MB/sec warmup 22 sec latency 54.004 ms 2024-05-09T08:44:08.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 15582 44.02 MB/sec warmup 23 sec latency 72.886 ms 2024-05-09T08:44:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:08 smithi160 ceph-mon[34070]: pgmap v803: 97 pgs: 97 active+clean; 354 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-05-09T08:44:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:08 smithi133 ceph-mon[29610]: pgmap v803: 97 pgs: 97 active+clean; 354 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-05-09T08:44:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:08 smithi188 ceph-mon[34645]: pgmap v803: 97 pgs: 97 active+clean; 354 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 468 op/s 2024-05-09T08:44:09.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 15843 42.85 MB/sec warmup 24 sec latency 236.426 ms 2024-05-09T08:44:10.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 16000 41.50 MB/sec warmup 25 sec latency 566.037 ms 2024-05-09T08:44:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:10 smithi160 ceph-mon[34070]: pgmap v804: 97 pgs: 97 active+clean; 392 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-05-09T08:44:10.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:10 smithi133 ceph-mon[29610]: pgmap v804: 97 pgs: 97 active+clean; 392 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-05-09T08:44:10.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:10 smithi188 ceph-mon[34645]: pgmap v804: 97 pgs: 97 active+clean; 392 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 527 op/s 2024-05-09T08:44:11.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 16634 41.39 MB/sec warmup 26 sec latency 74.857 ms 2024-05-09T08:44:12.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 17159 40.86 MB/sec warmup 27 sec latency 46.561 ms 2024-05-09T08:44:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:12 smithi160 ceph-mon[34070]: pgmap v805: 97 pgs: 97 active+clean; 478 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 21 MiB/s wr, 570 op/s 2024-05-09T08:44:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:12 smithi133 ceph-mon[29610]: pgmap v805: 97 pgs: 97 active+clean; 478 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 21 MiB/s wr, 570 op/s 2024-05-09T08:44:12.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:12 smithi188 ceph-mon[34645]: pgmap v805: 97 pgs: 97 active+clean; 478 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 21 MiB/s wr, 570 op/s 2024-05-09T08:44:13.338 INFO:tasks.workunit.client.0.smithi133.stdout: 10 17711 40.34 MB/sec warmup 28 sec latency 125.566 ms 2024-05-09T08:44:14.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 18384 40.23 MB/sec warmup 29 sec latency 107.427 ms 2024-05-09T08:44:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:14 smithi160 ceph-mon[34070]: pgmap v806: 97 pgs: 97 active+clean; 529 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 608 op/s 2024-05-09T08:44:14.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:14 smithi133 ceph-mon[29610]: pgmap v806: 97 pgs: 97 active+clean; 529 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 608 op/s 2024-05-09T08:44:14.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:14 smithi188 ceph-mon[34645]: pgmap v806: 97 pgs: 97 active+clean; 529 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 608 op/s 2024-05-09T08:44:15.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 18904 39.70 MB/sec warmup 30 sec latency 182.446 ms 2024-05-09T08:44:16.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 19598 39.58 MB/sec warmup 31 sec latency 83.456 ms 2024-05-09T08:44:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:16 smithi160 ceph-mon[34070]: pgmap v807: 97 pgs: 97 active+clean; 522 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 31 MiB/s wr, 862 op/s 2024-05-09T08:44:16.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:16 smithi133 ceph-mon[29610]: pgmap v807: 97 pgs: 97 active+clean; 522 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 31 MiB/s wr, 862 op/s 2024-05-09T08:44:16.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:16 smithi188 ceph-mon[34645]: pgmap v807: 97 pgs: 97 active+clean; 522 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 31 MiB/s wr, 862 op/s 2024-05-09T08:44:17.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 20219 39.51 MB/sec warmup 32 sec latency 48.237 ms 2024-05-09T08:44:18.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 20936 39.51 MB/sec warmup 33 sec latency 44.396 ms 2024-05-09T08:44:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:18 smithi160 ceph-mon[34070]: pgmap v808: 97 pgs: 97 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 713 op/s 2024-05-09T08:44:18.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:18 smithi133 ceph-mon[29610]: pgmap v808: 97 pgs: 97 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 713 op/s 2024-05-09T08:44:18.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:18 smithi188 ceph-mon[34645]: pgmap v808: 97 pgs: 97 active+clean; 518 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 27 MiB/s wr, 713 op/s 2024-05-09T08:44:19.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 21513 38.98 MB/sec warmup 34 sec latency 67.453 ms 2024-05-09T08:44:20.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 22111 39.19 MB/sec warmup 35 sec latency 235.432 ms 2024-05-09T08:44:20.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:20 smithi160 ceph-mon[34070]: pgmap v809: 97 pgs: 97 active+clean; 504 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 695 op/s 2024-05-09T08:44:20.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:20 smithi133 ceph-mon[29610]: pgmap v809: 97 pgs: 97 active+clean; 504 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 695 op/s 2024-05-09T08:44:20.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:20 smithi188 ceph-mon[34645]: pgmap v809: 97 pgs: 97 active+clean; 504 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 25 MiB/s wr, 695 op/s 2024-05-09T08:44:21.339 INFO:tasks.workunit.client.0.smithi133.stdout: 10 22751 38.92 MB/sec warmup 36 sec latency 88.257 ms 2024-05-09T08:44:22.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 23502 38.70 MB/sec warmup 37 sec latency 58.226 ms 2024-05-09T08:44:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:22 smithi160 ceph-mon[34070]: pgmap v810: 97 pgs: 97 active+clean; 507 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 675 op/s 2024-05-09T08:44:22.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:22 smithi133 ceph-mon[29610]: pgmap v810: 97 pgs: 97 active+clean; 507 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 675 op/s 2024-05-09T08:44:22.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:22 smithi188 ceph-mon[34645]: pgmap v810: 97 pgs: 97 active+clean; 507 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 675 op/s 2024-05-09T08:44:23.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 24214 38.97 MB/sec warmup 38 sec latency 57.373 ms 2024-05-09T08:44:24.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 24819 38.84 MB/sec warmup 39 sec latency 50.260 ms 2024-05-09T08:44:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:24 smithi160 ceph-mon[34070]: pgmap v811: 97 pgs: 97 active+clean; 510 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 607 op/s 2024-05-09T08:44:24.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:24 smithi133 ceph-mon[29610]: pgmap v811: 97 pgs: 97 active+clean; 510 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 607 op/s 2024-05-09T08:44:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:24 smithi188 ceph-mon[34645]: pgmap v811: 97 pgs: 97 active+clean; 510 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 607 op/s 2024-05-09T08:44:25.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 25151 38.24 MB/sec warmup 40 sec latency 288.885 ms 2024-05-09T08:44:26.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 25654 37.96 MB/sec warmup 41 sec latency 350.801 ms 2024-05-09T08:44:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:26 smithi160 ceph-mon[34070]: pgmap v812: 97 pgs: 97 active+clean; 516 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-09T08:44:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:26 smithi133 ceph-mon[29610]: pgmap v812: 97 pgs: 97 active+clean; 516 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-09T08:44:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:26 smithi188 ceph-mon[34645]: pgmap v812: 97 pgs: 97 active+clean; 516 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.3 MiB/s wr, 573 op/s 2024-05-09T08:44:27.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 26280 37.68 MB/sec warmup 42 sec latency 62.645 ms 2024-05-09T08:44:28.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 27104 37.98 MB/sec warmup 43 sec latency 69.499 ms 2024-05-09T08:44:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:28 smithi160 ceph-mon[34070]: pgmap v813: 97 pgs: 97 active+clean; 519 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-05-09T08:44:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:28 smithi133 ceph-mon[29610]: pgmap v813: 97 pgs: 97 active+clean; 519 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-05-09T08:44:28.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:28 smithi188 ceph-mon[34645]: pgmap v813: 97 pgs: 97 active+clean; 519 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.8 MiB/s wr, 384 op/s 2024-05-09T08:44:29.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 27878 38.17 MB/sec warmup 44 sec latency 47.157 ms 2024-05-09T08:44:30.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 28214 37.70 MB/sec warmup 45 sec latency 314.601 ms 2024-05-09T08:44:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:30 smithi160 ceph-mon[34070]: pgmap v814: 97 pgs: 97 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-05-09T08:44:30.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:30 smithi133 ceph-mon[29610]: pgmap v814: 97 pgs: 97 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-05-09T08:44:30.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:30 smithi188 ceph-mon[34645]: pgmap v814: 97 pgs: 97 active+clean; 521 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.3 MiB/s wr, 446 op/s 2024-05-09T08:44:31.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 28351 37.08 MB/sec warmup 46 sec latency 845.848 ms 2024-05-09T08:44:32.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 28824 36.75 MB/sec warmup 47 sec latency 148.702 ms 2024-05-09T08:44:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:32 smithi160 ceph-mon[34070]: pgmap v815: 97 pgs: 97 active+clean; 532 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 540 op/s 2024-05-09T08:44:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:32 smithi133 ceph-mon[29610]: pgmap v815: 97 pgs: 97 active+clean; 532 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 540 op/s 2024-05-09T08:44:32.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:32 smithi188 ceph-mon[34645]: pgmap v815: 97 pgs: 97 active+clean; 532 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.8 MiB/s wr, 540 op/s 2024-05-09T08:44:33.340 INFO:tasks.workunit.client.0.smithi133.stdout: 10 29320 36.56 MB/sec warmup 48 sec latency 69.866 ms 2024-05-09T08:44:34.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 29974 36.40 MB/sec warmup 49 sec latency 46.087 ms 2024-05-09T08:44:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:34 smithi160 ceph-mon[34070]: pgmap v816: 97 pgs: 97 active+clean; 551 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-05-09T08:44:34.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:34 smithi133 ceph-mon[29610]: pgmap v816: 97 pgs: 97 active+clean; 551 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-05-09T08:44:34.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:34 smithi188 ceph-mon[34645]: pgmap v816: 97 pgs: 97 active+clean; 551 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.7 MiB/s wr, 516 op/s 2024-05-09T08:44:35.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 30677 36.38 MB/sec warmup 50 sec latency 58.329 ms 2024-05-09T08:44:36.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 31435 36.73 MB/sec warmup 51 sec latency 59.946 ms 2024-05-09T08:44:36.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:36 smithi160 ceph-mon[34070]: pgmap v817: 97 pgs: 97 active+clean; 556 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-09T08:44:36.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:36 smithi133 ceph-mon[29610]: pgmap v817: 97 pgs: 97 active+clean; 556 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-09T08:44:36.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:36 smithi188 ceph-mon[34645]: pgmap v817: 97 pgs: 97 active+clean; 556 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-05-09T08:44:37.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 32142 36.68 MB/sec warmup 52 sec latency 109.199 ms 2024-05-09T08:44:38.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 32637 36.46 MB/sec warmup 53 sec latency 102.808 ms 2024-05-09T08:44:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:38 smithi160 ceph-mon[34070]: pgmap v818: 97 pgs: 97 active+clean; 539 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 697 op/s 2024-05-09T08:44:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:38 smithi133 ceph-mon[29610]: pgmap v818: 97 pgs: 97 active+clean; 539 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 697 op/s 2024-05-09T08:44:38.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:38 smithi188 ceph-mon[34645]: pgmap v818: 97 pgs: 97 active+clean; 539 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 697 op/s 2024-05-09T08:44:39.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 33234 36.42 MB/sec warmup 54 sec latency 59.932 ms 2024-05-09T08:44:40.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 33773 36.38 MB/sec warmup 55 sec latency 95.131 ms 2024-05-09T08:44:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:40 smithi160 ceph-mon[34070]: pgmap v819: 97 pgs: 97 active+clean; 526 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-05-09T08:44:40.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:40 smithi133 ceph-mon[29610]: pgmap v819: 97 pgs: 97 active+clean; 526 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-05-09T08:44:40.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:40 smithi188 ceph-mon[34645]: pgmap v819: 97 pgs: 97 active+clean; 526 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-05-09T08:44:41.341 INFO:tasks.workunit.client.0.smithi133.stdout: 10 34337 36.02 MB/sec warmup 56 sec latency 50.319 ms 2024-05-09T08:44:42.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 35071 36.01 MB/sec warmup 57 sec latency 102.683 ms 2024-05-09T08:44:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:42 smithi160 ceph-mon[34070]: pgmap v820: 97 pgs: 97 active+clean; 518 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-05-09T08:44:42.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:42 smithi133 ceph-mon[29610]: pgmap v820: 97 pgs: 97 active+clean; 518 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-05-09T08:44:42.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:42 smithi188 ceph-mon[34645]: pgmap v820: 97 pgs: 97 active+clean; 518 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 657 op/s 2024-05-09T08:44:43.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 35886 36.41 MB/sec warmup 58 sec latency 64.594 ms 2024-05-09T08:44:44.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 36533 36.36 MB/sec warmup 59 sec latency 79.774 ms 2024-05-09T08:44:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:44 smithi160 ceph-mon[34070]: pgmap v821: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 558 op/s 2024-05-09T08:44:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:44 smithi133 ceph-mon[29610]: pgmap v821: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 558 op/s 2024-05-09T08:44:44.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:44 smithi188 ceph-mon[34645]: pgmap v821: 97 pgs: 97 active+clean; 517 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.4 MiB/s wr, 558 op/s 2024-05-09T08:44:45.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 37070 36.23 MB/sec warmup 60 sec latency 85.725 ms 2024-05-09T08:44:46.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 37591 36.13 MB/sec warmup 61 sec latency 66.042 ms 2024-05-09T08:44:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:46 smithi160 ceph-mon[34070]: pgmap v822: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.0 MiB/s wr, 618 op/s 2024-05-09T08:44:46.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:46 smithi133 ceph-mon[29610]: pgmap v822: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.0 MiB/s wr, 618 op/s 2024-05-09T08:44:46.931 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:46 smithi188 ceph-mon[34645]: pgmap v822: 97 pgs: 97 active+clean; 528 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 8.0 MiB/s wr, 618 op/s 2024-05-09T08:44:47.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 38258 36.01 MB/sec warmup 62 sec latency 62.655 ms 2024-05-09T08:44:48.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 38932 36.02 MB/sec warmup 63 sec latency 61.167 ms 2024-05-09T08:44:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:48 smithi160 ceph-mon[34070]: pgmap v823: 97 pgs: 97 active+clean; 530 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 413 op/s 2024-05-09T08:44:48.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:48 smithi133 ceph-mon[29610]: pgmap v823: 97 pgs: 97 active+clean; 530 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 413 op/s 2024-05-09T08:44:48.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:48 smithi188 ceph-mon[34645]: pgmap v823: 97 pgs: 97 active+clean; 530 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 413 op/s 2024-05-09T08:44:49.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 39624 36.08 MB/sec warmup 64 sec latency 74.112 ms 2024-05-09T08:44:50.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 40290 36.05 MB/sec warmup 65 sec latency 160.038 ms 2024-05-09T08:44:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:50 smithi160 ceph-mon[34070]: pgmap v824: 97 pgs: 97 active+clean; 538 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-05-09T08:44:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:50 smithi133 ceph-mon[29610]: pgmap v824: 97 pgs: 97 active+clean; 538 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-05-09T08:44:50.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:50 smithi188 ceph-mon[34645]: pgmap v824: 97 pgs: 97 active+clean; 538 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.6 MiB/s wr, 401 op/s 2024-05-09T08:44:51.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 40785 35.91 MB/sec warmup 66 sec latency 360.048 ms 2024-05-09T08:44:52.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 41425 35.94 MB/sec warmup 67 sec latency 83.623 ms 2024-05-09T08:44:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:52 smithi160 ceph-mon[34070]: pgmap v825: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-09T08:44:52.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:52 smithi133 ceph-mon[29610]: pgmap v825: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-09T08:44:52.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:52 smithi188 ceph-mon[34645]: pgmap v825: 97 pgs: 97 active+clean; 545 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-05-09T08:44:53.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 42088 35.95 MB/sec warmup 68 sec latency 100.628 ms 2024-05-09T08:44:53.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:53 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:44:53.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:53 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:44:53.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:53 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:44:54.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 42579 35.76 MB/sec warmup 69 sec latency 58.131 ms 2024-05-09T08:44:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:54 smithi160 ceph-mon[34070]: pgmap v826: 97 pgs: 97 active+clean; 549 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-09T08:44:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:54 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:44:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:54 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:44:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:54 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:44:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:54 smithi133 ceph-mon[29610]: pgmap v826: 97 pgs: 97 active+clean; 549 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-09T08:44:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:54 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:44:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:54 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:44:54.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:54 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:44:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:54 smithi188 ceph-mon[34645]: pgmap v826: 97 pgs: 97 active+clean; 549 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-05-09T08:44:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:54 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:44:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:54 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:44:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:54 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:44:55.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 43204 35.78 MB/sec warmup 70 sec latency 117.097 ms 2024-05-09T08:44:56.342 INFO:tasks.workunit.client.0.smithi133.stdout: 10 43719 35.61 MB/sec warmup 71 sec latency 159.861 ms 2024-05-09T08:44:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:56 smithi160 ceph-mon[34070]: pgmap v827: 97 pgs: 97 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-05-09T08:44:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:56 smithi133 ceph-mon[29610]: pgmap v827: 97 pgs: 97 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-05-09T08:44:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:56 smithi188 ceph-mon[34645]: pgmap v827: 97 pgs: 97 active+clean; 554 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.9 MiB/s wr, 445 op/s 2024-05-09T08:44:57.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 44324 35.52 MB/sec warmup 72 sec latency 173.053 ms 2024-05-09T08:44:58.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 44834 35.50 MB/sec warmup 73 sec latency 307.699 ms 2024-05-09T08:44:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:44:58 smithi160 ceph-mon[34070]: pgmap v828: 97 pgs: 97 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-05-09T08:44:58.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:44:58 smithi133 ceph-mon[29610]: pgmap v828: 97 pgs: 97 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-05-09T08:44:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:44:58 smithi188 ceph-mon[34645]: pgmap v828: 97 pgs: 97 active+clean; 563 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 408 op/s 2024-05-09T08:44:59.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 45490 35.48 MB/sec warmup 74 sec latency 61.641 ms 2024-05-09T08:45:00.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 46218 35.49 MB/sec warmup 75 sec latency 62.259 ms 2024-05-09T08:45:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:00 smithi160 ceph-mon[34070]: pgmap v829: 97 pgs: 97 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-09T08:45:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:00 smithi133 ceph-mon[29610]: pgmap v829: 97 pgs: 97 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-09T08:45:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:00 smithi188 ceph-mon[34645]: pgmap v829: 97 pgs: 97 active+clean; 565 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-05-09T08:45:01.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 46833 35.50 MB/sec warmup 76 sec latency 80.683 ms 2024-05-09T08:45:02.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 46851 35.06 MB/sec warmup 77 sec latency 1019.635 ms 2024-05-09T08:45:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:02 smithi160 ceph-mon[34070]: pgmap v830: 97 pgs: 97 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 510 op/s 2024-05-09T08:45:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:02 smithi133 ceph-mon[29610]: pgmap v830: 97 pgs: 97 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 510 op/s 2024-05-09T08:45:03.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:02 smithi188 ceph-mon[34645]: pgmap v830: 97 pgs: 97 active+clean; 569 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.8 MiB/s wr, 510 op/s 2024-05-09T08:45:03.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 47028 34.79 MB/sec warmup 78 sec latency 1281.429 ms 2024-05-09T08:45:04.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 47319 34.53 MB/sec warmup 79 sec latency 421.323 ms 2024-05-09T08:45:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:04 smithi160 ceph-mon[34070]: pgmap v831: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 486 op/s 2024-05-09T08:45:05.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:04 smithi133 ceph-mon[29610]: pgmap v831: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 486 op/s 2024-05-09T08:45:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:04 smithi188 ceph-mon[34645]: pgmap v831: 97 pgs: 97 active+clean; 580 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.0 MiB/s wr, 486 op/s 2024-05-09T08:45:05.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 47954 34.48 MB/sec warmup 80 sec latency 45.901 ms 2024-05-09T08:45:06.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 48477 34.33 MB/sec warmup 81 sec latency 95.596 ms 2024-05-09T08:45:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:06 smithi160 ceph-mon[34070]: pgmap v832: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:45:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:06 smithi133 ceph-mon[29610]: pgmap v832: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:45:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:06 smithi188 ceph-mon[34645]: pgmap v832: 97 pgs: 97 active+clean; 575 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:45:07.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 49159 34.42 MB/sec warmup 82 sec latency 77.466 ms 2024-05-09T08:45:08.343 INFO:tasks.workunit.client.0.smithi133.stdout: 10 49872 34.39 MB/sec warmup 83 sec latency 95.759 ms 2024-05-09T08:45:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:08 smithi160 ceph-mon[34070]: pgmap v833: 97 pgs: 97 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-05-09T08:45:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:08 smithi133 ceph-mon[29610]: pgmap v833: 97 pgs: 97 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-05-09T08:45:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:08 smithi188 ceph-mon[34645]: pgmap v833: 97 pgs: 97 active+clean; 570 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 719 op/s 2024-05-09T08:45:09.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 50428 34.46 MB/sec warmup 84 sec latency 255.489 ms 2024-05-09T08:45:10.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 50972 34.41 MB/sec warmup 85 sec latency 58.107 ms 2024-05-09T08:45:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:10 smithi160 ceph-mon[34070]: pgmap v834: 97 pgs: 97 active+clean; 540 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 792 op/s 2024-05-09T08:45:11.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:10 smithi133 ceph-mon[29610]: pgmap v834: 97 pgs: 97 active+clean; 540 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 792 op/s 2024-05-09T08:45:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:10 smithi188 ceph-mon[34645]: pgmap v834: 97 pgs: 97 active+clean; 540 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 792 op/s 2024-05-09T08:45:11.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 51520 34.35 MB/sec warmup 86 sec latency 77.882 ms 2024-05-09T08:45:12.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 52191 34.28 MB/sec warmup 87 sec latency 46.932 ms 2024-05-09T08:45:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:12 smithi160 ceph-mon[34070]: pgmap v835: 97 pgs: 97 active+clean; 544 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T08:45:13.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:12 smithi133 ceph-mon[29610]: pgmap v835: 97 pgs: 97 active+clean; 544 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T08:45:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:12 smithi188 ceph-mon[34645]: pgmap v835: 97 pgs: 97 active+clean; 544 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 779 op/s 2024-05-09T08:45:13.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 52578 34.18 MB/sec warmup 88 sec latency 194.728 ms 2024-05-09T08:45:14.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 53341 34.26 MB/sec warmup 89 sec latency 46.125 ms 2024-05-09T08:45:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:14 smithi160 ceph-mon[34070]: pgmap v836: 97 pgs: 97 active+clean; 546 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 655 op/s 2024-05-09T08:45:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:14 smithi133 ceph-mon[29610]: pgmap v836: 97 pgs: 97 active+clean; 546 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 655 op/s 2024-05-09T08:45:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:14 smithi188 ceph-mon[34645]: pgmap v836: 97 pgs: 97 active+clean; 546 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 655 op/s 2024-05-09T08:45:15.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 54051 34.24 MB/sec warmup 90 sec latency 46.875 ms 2024-05-09T08:45:16.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 54806 34.38 MB/sec warmup 91 sec latency 57.229 ms 2024-05-09T08:45:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:16 smithi160 ceph-mon[34070]: pgmap v837: 97 pgs: 97 active+clean; 555 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 701 op/s 2024-05-09T08:45:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:16 smithi133 ceph-mon[29610]: pgmap v837: 97 pgs: 97 active+clean; 555 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 701 op/s 2024-05-09T08:45:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:16 smithi188 ceph-mon[34645]: pgmap v837: 97 pgs: 97 active+clean; 555 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 701 op/s 2024-05-09T08:45:17.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 55326 34.34 MB/sec warmup 92 sec latency 193.272 ms 2024-05-09T08:45:18.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 55779 34.26 MB/sec warmup 93 sec latency 210.494 ms 2024-05-09T08:45:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:18 smithi160 ceph-mon[34070]: pgmap v838: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 522 op/s 2024-05-09T08:45:19.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:18 smithi133 ceph-mon[29610]: pgmap v838: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 522 op/s 2024-05-09T08:45:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:18 smithi188 ceph-mon[34645]: pgmap v838: 97 pgs: 97 active+clean; 562 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.3 MiB/s wr, 522 op/s 2024-05-09T08:45:19.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 56318 34.12 MB/sec warmup 94 sec latency 75.091 ms 2024-05-09T08:45:20.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 57053 34.12 MB/sec warmup 95 sec latency 54.924 ms 2024-05-09T08:45:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:20 smithi160 ceph-mon[34070]: pgmap v839: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-05-09T08:45:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:20 smithi133 ceph-mon[29610]: pgmap v839: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-05-09T08:45:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:20 smithi188 ceph-mon[34645]: pgmap v839: 97 pgs: 97 active+clean; 559 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-05-09T08:45:21.344 INFO:tasks.workunit.client.0.smithi133.stdout: 10 57814 34.24 MB/sec warmup 96 sec latency 40.114 ms 2024-05-09T08:45:22.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 58403 34.22 MB/sec warmup 97 sec latency 160.453 ms 2024-05-09T08:45:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:22 smithi160 ceph-mon[34070]: pgmap v840: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-05-09T08:45:23.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:22 smithi133 ceph-mon[29610]: pgmap v840: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-05-09T08:45:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:22 smithi188 ceph-mon[34645]: pgmap v840: 97 pgs: 97 active+clean; 544 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 418 op/s 2024-05-09T08:45:23.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 58548 33.94 MB/sec warmup 98 sec latency 500.192 ms 2024-05-09T08:45:24.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 59069 33.92 MB/sec warmup 99 sec latency 108.571 ms 2024-05-09T08:45:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:24 smithi160 ceph-mon[34070]: pgmap v841: 97 pgs: 97 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-09T08:45:25.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:24 smithi133 ceph-mon[29610]: pgmap v841: 97 pgs: 97 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-09T08:45:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:24 smithi188 ceph-mon[34645]: pgmap v841: 97 pgs: 97 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 469 op/s 2024-05-09T08:45:25.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 59750 33.99 MB/sec warmup 100 sec latency 78.319 ms 2024-05-09T08:45:26.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 60262 33.89 MB/sec warmup 101 sec latency 91.945 ms 2024-05-09T08:45:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:26 smithi160 ceph-mon[34070]: pgmap v842: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-05-09T08:45:27.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:26 smithi133 ceph-mon[29610]: pgmap v842: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-05-09T08:45:27.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:26 smithi188 ceph-mon[34645]: pgmap v842: 97 pgs: 97 active+clean; 554 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 626 op/s 2024-05-09T08:45:27.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 60818 33.83 MB/sec warmup 102 sec latency 64.729 ms 2024-05-09T08:45:28.345 INFO:tasks.workunit.client.0.smithi133.stdout: 10 61426 33.84 MB/sec warmup 103 sec latency 95.573 ms 2024-05-09T08:45:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:28 smithi160 ceph-mon[34070]: pgmap v843: 97 pgs: 97 active+clean; 558 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-09T08:45:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:28 smithi133 ceph-mon[29610]: pgmap v843: 97 pgs: 97 active+clean; 558 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-09T08:45:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:28 smithi188 ceph-mon[34645]: pgmap v843: 97 pgs: 97 active+clean; 558 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 610 op/s 2024-05-09T08:45:29.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 62155 33.83 MB/sec warmup 104 sec latency 56.441 ms 2024-05-09T08:45:30.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 62881 33.89 MB/sec warmup 105 sec latency 52.233 ms 2024-05-09T08:45:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:30 smithi160 ceph-mon[34070]: pgmap v844: 97 pgs: 97 active+clean; 553 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-09T08:45:31.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:30 smithi133 ceph-mon[29610]: pgmap v844: 97 pgs: 97 active+clean; 553 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-09T08:45:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:30 smithi188 ceph-mon[34645]: pgmap v844: 97 pgs: 97 active+clean; 553 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 646 op/s 2024-05-09T08:45:31.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 63575 34.03 MB/sec warmup 106 sec latency 53.475 ms 2024-05-09T08:45:32.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 64084 33.95 MB/sec warmup 107 sec latency 96.150 ms 2024-05-09T08:45:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:32 smithi160 ceph-mon[34070]: pgmap v845: 97 pgs: 97 active+clean; 560 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:45:33.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:32 smithi133 ceph-mon[29610]: pgmap v845: 97 pgs: 97 active+clean; 560 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:45:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:32 smithi188 ceph-mon[34645]: pgmap v845: 97 pgs: 97 active+clean; 560 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:45:33.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 64585 33.82 MB/sec warmup 108 sec latency 105.410 ms 2024-05-09T08:45:34.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 65086 33.77 MB/sec warmup 109 sec latency 393.985 ms 2024-05-09T08:45:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:34 smithi160 ceph-mon[34070]: pgmap v846: 97 pgs: 97 active+clean; 561 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-05-09T08:45:35.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:34 smithi133 ceph-mon[29610]: pgmap v846: 97 pgs: 97 active+clean; 561 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-05-09T08:45:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:34 smithi188 ceph-mon[34645]: pgmap v846: 97 pgs: 97 active+clean; 561 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 602 op/s 2024-05-09T08:45:35.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 65787 33.77 MB/sec warmup 110 sec latency 53.772 ms 2024-05-09T08:45:36.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 66421 33.80 MB/sec warmup 111 sec latency 59.847 ms 2024-05-09T08:45:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:36 smithi160 ceph-mon[34070]: pgmap v847: 97 pgs: 97 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-05-09T08:45:37.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:36 smithi133 ceph-mon[29610]: pgmap v847: 97 pgs: 97 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-05-09T08:45:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:36 smithi188 ceph-mon[34645]: pgmap v847: 97 pgs: 97 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 635 op/s 2024-05-09T08:45:37.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 67145 33.87 MB/sec warmup 112 sec latency 42.822 ms 2024-05-09T08:45:38.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 67625 33.83 MB/sec warmup 113 sec latency 220.252 ms 2024-05-09T08:45:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:38 smithi160 ceph-mon[34070]: pgmap v848: 97 pgs: 97 active+clean; 574 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 476 op/s 2024-05-09T08:45:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:38 smithi133 ceph-mon[29610]: pgmap v848: 97 pgs: 97 active+clean; 574 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 476 op/s 2024-05-09T08:45:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:38 smithi188 ceph-mon[34645]: pgmap v848: 97 pgs: 97 active+clean; 574 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 476 op/s 2024-05-09T08:45:39.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 68157 33.75 MB/sec warmup 114 sec latency 272.304 ms 2024-05-09T08:45:40.346 INFO:tasks.workunit.client.0.smithi133.stdout: 10 68732 33.76 MB/sec warmup 115 sec latency 166.704 ms 2024-05-09T08:45:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:40 smithi160 ceph-mon[34070]: pgmap v849: 97 pgs: 97 active+clean; 579 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-05-09T08:45:41.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:40 smithi133 ceph-mon[29610]: pgmap v849: 97 pgs: 97 active+clean; 579 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-05-09T08:45:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:40 smithi188 ceph-mon[34645]: pgmap v849: 97 pgs: 97 active+clean; 579 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.3 MiB/s wr, 448 op/s 2024-05-09T08:45:41.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 69353 33.74 MB/sec warmup 116 sec latency 69.090 ms 2024-05-09T08:45:42.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 70062 33.70 MB/sec warmup 117 sec latency 39.907 ms 2024-05-09T08:45:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:42 smithi160 ceph-mon[34070]: pgmap v850: 97 pgs: 97 active+clean; 572 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-09T08:45:43.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:42 smithi133 ceph-mon[29610]: pgmap v850: 97 pgs: 97 active+clean; 572 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-09T08:45:43.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:42 smithi188 ceph-mon[34645]: pgmap v850: 97 pgs: 97 active+clean; 572 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.2 MiB/s wr, 471 op/s 2024-05-09T08:45:43.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 70708 33.77 MB/sec warmup 118 sec latency 111.313 ms 2024-05-09T08:45:44.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 70837 33.52 MB/sec warmup 119 sec latency 616.839 ms 2024-05-09T08:45:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:44 smithi160 ceph-mon[34070]: pgmap v851: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 432 op/s 2024-05-09T08:45:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:44 smithi133 ceph-mon[29610]: pgmap v851: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 432 op/s 2024-05-09T08:45:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:44 smithi188 ceph-mon[34645]: pgmap v851: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 432 op/s 2024-05-09T08:45:46.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 72004 35.67 MB/sec execute 1 sec latency 103.205 ms 2024-05-09T08:45:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:46 smithi160 ceph-mon[34070]: pgmap v852: 97 pgs: 97 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:45:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:46 smithi133 ceph-mon[29610]: pgmap v852: 97 pgs: 97 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:45:47.194 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:46 smithi188 ceph-mon[34645]: pgmap v852: 97 pgs: 97 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:45:47.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 72545 29.48 MB/sec execute 2 sec latency 63.711 ms 2024-05-09T08:45:48.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 73150 30.91 MB/sec execute 3 sec latency 75.761 ms 2024-05-09T08:45:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:48 smithi160 ceph-mon[34070]: pgmap v853: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:45:49.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:48 smithi133 ceph-mon[29610]: pgmap v853: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:45:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:48 smithi188 ceph-mon[34645]: pgmap v853: 97 pgs: 97 active+clean; 603 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:45:49.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 73692 29.41 MB/sec execute 4 sec latency 364.407 ms 2024-05-09T08:45:50.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 74300 32.04 MB/sec execute 5 sec latency 51.179 ms 2024-05-09T08:45:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:50 smithi160 ceph-mon[34070]: pgmap v854: 97 pgs: 97 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-09T08:45:51.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:50 smithi133 ceph-mon[29610]: pgmap v854: 97 pgs: 97 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-09T08:45:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:50 smithi188 ceph-mon[34645]: pgmap v854: 97 pgs: 97 active+clean; 577 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 620 op/s 2024-05-09T08:45:51.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 74843 30.77 MB/sec execute 6 sec latency 88.752 ms 2024-05-09T08:45:52.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 75428 30.57 MB/sec execute 7 sec latency 62.124 ms 2024-05-09T08:45:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:52 smithi160 ceph-mon[34070]: pgmap v855: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-09T08:45:53.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:52 smithi133 ceph-mon[29610]: pgmap v855: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-09T08:45:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:52 smithi188 ceph-mon[34645]: pgmap v855: 97 pgs: 97 active+clean; 568 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-09T08:45:53.347 INFO:tasks.workunit.client.0.smithi133.stdout: 10 76030 32.06 MB/sec execute 8 sec latency 143.436 ms 2024-05-09T08:45:54.348 INFO:tasks.workunit.client.0.smithi133.stdout: 10 76641 30.59 MB/sec execute 9 sec latency 94.049 ms 2024-05-09T08:45:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:54 smithi160 ceph-mon[34070]: pgmap v856: 97 pgs: 97 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T08:45:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:54 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:45:55.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:54 smithi133 ceph-mon[29610]: pgmap v856: 97 pgs: 97 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T08:45:55.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:54 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:45:55.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:54 smithi188 ceph-mon[34645]: pgmap v856: 97 pgs: 97 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 594 op/s 2024-05-09T08:45:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:54 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:45:55.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 77241 31.09 MB/sec execute 10 sec latency 84.274 ms 2024-05-09T08:45:56.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 77902 31.98 MB/sec execute 11 sec latency 80.033 ms 2024-05-09T08:45:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:56 smithi160 ceph-mon[34070]: pgmap v857: 97 pgs: 97 active+clean; 580 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-09T08:45:57.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:56 smithi133 ceph-mon[29610]: pgmap v857: 97 pgs: 97 active+clean; 580 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-09T08:45:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:56 smithi188 ceph-mon[34645]: pgmap v857: 97 pgs: 97 active+clean; 580 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-05-09T08:45:57.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 78482 31.93 MB/sec execute 12 sec latency 57.765 ms 2024-05-09T08:45:58.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 79030 31.67 MB/sec execute 13 sec latency 97.228 ms 2024-05-09T08:45:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:45:58 smithi160 ceph-mon[34070]: pgmap v858: 97 pgs: 97 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-09T08:45:59.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:45:58 smithi133 ceph-mon[29610]: pgmap v858: 97 pgs: 97 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-09T08:45:59.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:45:58 smithi188 ceph-mon[34645]: pgmap v858: 97 pgs: 97 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-09T08:45:59.349 INFO:tasks.workunit.client.0.smithi133.stdout: 10 79545 31.11 MB/sec execute 14 sec latency 89.899 ms 2024-05-09T08:46:00.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 80137 31.80 MB/sec execute 15 sec latency 105.023 ms 2024-05-09T08:46:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:00 smithi160 ceph-mon[34070]: pgmap v859: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2024-05-09T08:46:01.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:00 smithi133 ceph-mon[29610]: pgmap v859: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2024-05-09T08:46:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:00 smithi188 ceph-mon[34645]: pgmap v859: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 505 op/s 2024-05-09T08:46:01.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 80780 31.45 MB/sec execute 16 sec latency 72.204 ms 2024-05-09T08:46:02.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:46:02.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:46:02.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:01 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:46:02.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:46:02.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:46:02.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:01 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:46:02.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:46:02.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:46:02.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:01 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:46:02.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 81450 31.31 MB/sec execute 17 sec latency 50.643 ms 2024-05-09T08:46:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:02 smithi160 ceph-mon[34070]: pgmap v860: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-05-09T08:46:03.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:02 smithi133 ceph-mon[29610]: pgmap v860: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-05-09T08:46:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:02 smithi188 ceph-mon[34645]: pgmap v860: 97 pgs: 97 active+clean; 570 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-05-09T08:46:03.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 82273 32.95 MB/sec execute 18 sec latency 45.980 ms 2024-05-09T08:46:04.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 82827 32.64 MB/sec execute 19 sec latency 61.446 ms 2024-05-09T08:46:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:04 smithi160 ceph-mon[34070]: pgmap v861: 97 pgs: 97 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-05-09T08:46:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:04 smithi133 ceph-mon[29610]: pgmap v861: 97 pgs: 97 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-05-09T08:46:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:04 smithi188 ceph-mon[34645]: pgmap v861: 97 pgs: 97 active+clean; 577 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-05-09T08:46:05.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 83281 32.07 MB/sec execute 20 sec latency 69.322 ms 2024-05-09T08:46:06.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 83862 31.87 MB/sec execute 21 sec latency 102.292 ms 2024-05-09T08:46:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:06 smithi160 ceph-mon[34070]: pgmap v862: 97 pgs: 97 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:46:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:06 smithi133 ceph-mon[29610]: pgmap v862: 97 pgs: 97 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:46:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:06 smithi188 ceph-mon[34645]: pgmap v862: 97 pgs: 97 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:46:07.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 84567 32.15 MB/sec execute 22 sec latency 74.863 ms 2024-05-09T08:46:08.350 INFO:tasks.workunit.client.0.smithi133.stdout: 10 85267 31.92 MB/sec execute 23 sec latency 40.220 ms 2024-05-09T08:46:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:08 smithi160 ceph-mon[34070]: pgmap v863: 97 pgs: 97 active+clean; 583 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-05-09T08:46:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:08 smithi133 ceph-mon[29610]: pgmap v863: 97 pgs: 97 active+clean; 583 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-05-09T08:46:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:08 smithi188 ceph-mon[34645]: pgmap v863: 97 pgs: 97 active+clean; 583 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-05-09T08:46:09.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 86008 32.91 MB/sec execute 24 sec latency 62.337 ms 2024-05-09T08:46:10.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 86346 32.42 MB/sec execute 25 sec latency 300.021 ms 2024-05-09T08:46:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:10 smithi160 ceph-mon[34070]: pgmap v864: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-09T08:46:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:10 smithi133 ceph-mon[29610]: pgmap v864: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-09T08:46:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:10 smithi188 ceph-mon[34645]: pgmap v864: 97 pgs: 97 active+clean; 593 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 414 op/s 2024-05-09T08:46:11.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 86956 32.16 MB/sec execute 26 sec latency 91.400 ms 2024-05-09T08:46:12.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 87428 31.94 MB/sec execute 27 sec latency 59.198 ms 2024-05-09T08:46:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:12 smithi160 ceph-mon[34070]: pgmap v865: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-05-09T08:46:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:12 smithi133 ceph-mon[29610]: pgmap v865: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-05-09T08:46:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:12 smithi188 ceph-mon[34645]: pgmap v865: 97 pgs: 97 active+clean; 601 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.0 MiB/s wr, 416 op/s 2024-05-09T08:46:13.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 88039 32.18 MB/sec execute 28 sec latency 88.408 ms 2024-05-09T08:46:14.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 88439 31.61 MB/sec execute 29 sec latency 395.468 ms 2024-05-09T08:46:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:14 smithi160 ceph-mon[34070]: pgmap v866: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 428 op/s 2024-05-09T08:46:15.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:14 smithi133 ceph-mon[29610]: pgmap v866: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 428 op/s 2024-05-09T08:46:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:14 smithi188 ceph-mon[34645]: pgmap v866: 97 pgs: 97 active+clean; 602 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 428 op/s 2024-05-09T08:46:15.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 88533 30.62 MB/sec execute 30 sec latency 884.769 ms 2024-05-09T08:46:16.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 88862 29.98 MB/sec execute 31 sec latency 1128.025 ms 2024-05-09T08:46:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:16 smithi160 ceph-mon[34070]: pgmap v867: 97 pgs: 97 active+clean; 609 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2024-05-09T08:46:17.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:16 smithi133 ceph-mon[29610]: pgmap v867: 97 pgs: 97 active+clean; 609 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2024-05-09T08:46:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:16 smithi188 ceph-mon[34645]: pgmap v867: 97 pgs: 97 active+clean; 609 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2024-05-09T08:46:17.359 INFO:tasks.workunit.client.0.smithi133.stdout: 10 89341 29.88 MB/sec execute 32 sec latency 119.735 ms 2024-05-09T08:46:18.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 89973 30.18 MB/sec execute 33 sec latency 71.706 ms 2024-05-09T08:46:19.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:18 smithi133 ceph-mon[29610]: pgmap v868: 97 pgs: 97 active+clean; 625 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.3 MiB/s wr, 510 op/s 2024-05-09T08:46:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:18 smithi188 ceph-mon[34645]: pgmap v868: 97 pgs: 97 active+clean; 625 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.3 MiB/s wr, 510 op/s 2024-05-09T08:46:19.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 90691 30.68 MB/sec execute 34 sec latency 55.660 ms 2024-05-09T08:46:19.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:18 smithi160 ceph-mon[34070]: pgmap v868: 97 pgs: 97 active+clean; 625 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.3 MiB/s wr, 510 op/s 2024-05-09T08:46:20.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 91280 30.58 MB/sec execute 35 sec latency 59.289 ms 2024-05-09T08:46:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:20 smithi133 ceph-mon[29610]: pgmap v869: 97 pgs: 97 active+clean; 616 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 720 op/s 2024-05-09T08:46:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:20 smithi188 ceph-mon[34645]: pgmap v869: 97 pgs: 97 active+clean; 616 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 720 op/s 2024-05-09T08:46:21.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 91856 30.59 MB/sec execute 36 sec latency 117.342 ms 2024-05-09T08:46:21.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:20 smithi160 ceph-mon[34070]: pgmap v869: 97 pgs: 97 active+clean; 616 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 14 MiB/s wr, 720 op/s 2024-05-09T08:46:22.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 92504 30.80 MB/sec execute 37 sec latency 43.974 ms 2024-05-09T08:46:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:22 smithi133 ceph-mon[29610]: pgmap v870: 97 pgs: 97 active+clean; 597 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T08:46:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:22 smithi188 ceph-mon[34645]: pgmap v870: 97 pgs: 97 active+clean; 597 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T08:46:23.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 93140 30.74 MB/sec execute 38 sec latency 45.882 ms 2024-05-09T08:46:23.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:22 smithi160 ceph-mon[34070]: pgmap v870: 97 pgs: 97 active+clean; 597 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 16 MiB/s wr, 758 op/s 2024-05-09T08:46:24.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 93717 30.75 MB/sec execute 39 sec latency 73.762 ms 2024-05-09T08:46:25.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:24 smithi133 ceph-mon[29610]: pgmap v871: 97 pgs: 97 active+clean; 574 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 766 op/s 2024-05-09T08:46:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:24 smithi188 ceph-mon[34645]: pgmap v871: 97 pgs: 97 active+clean; 574 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 766 op/s 2024-05-09T08:46:25.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 94412 30.90 MB/sec execute 40 sec latency 53.492 ms 2024-05-09T08:46:25.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:24 smithi160 ceph-mon[34070]: pgmap v871: 97 pgs: 97 active+clean; 574 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 16 MiB/s wr, 766 op/s 2024-05-09T08:46:26.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 94993 31.11 MB/sec execute 41 sec latency 87.175 ms 2024-05-09T08:46:26.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:25 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:26.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:25 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:26.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:25 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:27.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 95591 31.01 MB/sec execute 42 sec latency 93.801 ms 2024-05-09T08:46:27.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:26 smithi160 ceph-mon[34070]: pgmap v872: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-05-09T08:46:27.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:26 smithi133 ceph-mon[29610]: pgmap v872: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-05-09T08:46:27.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:26 smithi188 ceph-mon[34645]: pgmap v872: 97 pgs: 97 active+clean; 583 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-05-09T08:46:28.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 96155 31.06 MB/sec execute 43 sec latency 63.924 ms 2024-05-09T08:46:29.360 INFO:tasks.workunit.client.0.smithi133.stdout: 10 96804 31.15 MB/sec execute 44 sec latency 67.285 ms 2024-05-09T08:46:29.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:28 smithi160 ceph-mon[34070]: pgmap v873: 97 pgs: 97 active+clean; 586 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-09T08:46:29.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:28 smithi133 ceph-mon[29610]: pgmap v873: 97 pgs: 97 active+clean; 586 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-09T08:46:29.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:28 smithi188 ceph-mon[34645]: pgmap v873: 97 pgs: 97 active+clean; 586 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-05-09T08:46:30.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 97366 31.02 MB/sec execute 45 sec latency 54.240 ms 2024-05-09T08:46:31.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 98017 31.25 MB/sec execute 46 sec latency 43.869 ms 2024-05-09T08:46:31.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:30 smithi160 ceph-mon[34070]: pgmap v874: 97 pgs: 97 active+clean; 590 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-09T08:46:31.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:30 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:31.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:30 smithi133 ceph-mon[29610]: pgmap v874: 97 pgs: 97 active+clean; 590 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-09T08:46:31.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:30 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:31.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:30 smithi188 ceph-mon[34645]: pgmap v874: 97 pgs: 97 active+clean; 590 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 668 op/s 2024-05-09T08:46:31.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:30 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:46:32.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 98639 31.23 MB/sec execute 47 sec latency 68.083 ms 2024-05-09T08:46:32.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:32 smithi160 ceph-mon[34070]: pgmap v875: 97 pgs: 97 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-05-09T08:46:32.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:32 smithi133 ceph-mon[29610]: pgmap v875: 97 pgs: 97 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-05-09T08:46:32.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:32 smithi188 ceph-mon[34645]: pgmap v875: 97 pgs: 97 active+clean; 596 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 526 op/s 2024-05-09T08:46:33.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 99260 31.19 MB/sec execute 48 sec latency 61.919 ms 2024-05-09T08:46:34.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 99738 30.91 MB/sec execute 49 sec latency 348.033 ms 2024-05-09T08:46:34.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:34 smithi160 ceph-mon[34070]: pgmap v876: 97 pgs: 97 active+clean; 602 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 416 op/s 2024-05-09T08:46:34.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:34 smithi133 ceph-mon[29610]: pgmap v876: 97 pgs: 97 active+clean; 602 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 416 op/s 2024-05-09T08:46:34.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:34 smithi188 ceph-mon[34645]: pgmap v876: 97 pgs: 97 active+clean; 602 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 416 op/s 2024-05-09T08:46:35.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 100425 31.31 MB/sec execute 50 sec latency 56.575 ms 2024-05-09T08:46:36.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 100899 31.23 MB/sec execute 51 sec latency 188.324 ms 2024-05-09T08:46:36.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:36 smithi160 ceph-mon[34070]: pgmap v877: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-05-09T08:46:36.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:36 smithi133 ceph-mon[29610]: pgmap v877: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-05-09T08:46:36.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:36 smithi188 ceph-mon[34645]: pgmap v877: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-05-09T08:46:37.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 101367 31.08 MB/sec execute 52 sec latency 174.482 ms 2024-05-09T08:46:38.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 101910 31.11 MB/sec execute 53 sec latency 55.340 ms 2024-05-09T08:46:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:38 smithi160 ceph-mon[34070]: pgmap v878: 97 pgs: 97 active+clean; 614 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 362 op/s 2024-05-09T08:46:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:38 smithi133 ceph-mon[29610]: pgmap v878: 97 pgs: 97 active+clean; 614 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 362 op/s 2024-05-09T08:46:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:38 smithi188 ceph-mon[34645]: pgmap v878: 97 pgs: 97 active+clean; 614 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 362 op/s 2024-05-09T08:46:39.361 INFO:tasks.workunit.client.0.smithi133.stdout: 10 102399 30.98 MB/sec execute 54 sec latency 138.181 ms 2024-05-09T08:46:40.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 103032 30.92 MB/sec execute 55 sec latency 58.011 ms 2024-05-09T08:46:40.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:40 smithi160 ceph-mon[34070]: pgmap v879: 97 pgs: 97 active+clean; 607 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-09T08:46:40.678 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:40 smithi133 ceph-mon[29610]: pgmap v879: 97 pgs: 97 active+clean; 607 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-09T08:46:40.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:40 smithi188 ceph-mon[34645]: pgmap v879: 97 pgs: 97 active+clean; 607 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.0 MiB/s wr, 441 op/s 2024-05-09T08:46:41.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 103171 30.50 MB/sec execute 56 sec latency 400.520 ms 2024-05-09T08:46:42.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 103328 30.08 MB/sec execute 57 sec latency 531.790 ms 2024-05-09T08:46:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:42 smithi160 ceph-mon[34070]: pgmap v880: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 487 op/s 2024-05-09T08:46:42.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:42 smithi133 ceph-mon[29610]: pgmap v880: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 487 op/s 2024-05-09T08:46:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:42 smithi188 ceph-mon[34645]: pgmap v880: 97 pgs: 97 active+clean; 608 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.8 MiB/s wr, 487 op/s 2024-05-09T08:46:43.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 103911 30.08 MB/sec execute 58 sec latency 76.706 ms 2024-05-09T08:46:44.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 104594 30.23 MB/sec execute 59 sec latency 117.170 ms 2024-05-09T08:46:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:44 smithi160 ceph-mon[34070]: pgmap v881: 97 pgs: 97 active+clean; 617 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-05-09T08:46:44.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:44 smithi133 ceph-mon[29610]: pgmap v881: 97 pgs: 97 active+clean; 617 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-05-09T08:46:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:44 smithi188 ceph-mon[34645]: pgmap v881: 97 pgs: 97 active+clean; 617 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-05-09T08:46:45.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 105338 30.46 MB/sec execute 60 sec latency 76.034 ms 2024-05-09T08:46:46.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 105922 30.58 MB/sec execute 61 sec latency 59.888 ms 2024-05-09T08:46:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:46 smithi160 ceph-mon[34070]: pgmap v882: 97 pgs: 97 active+clean; 618 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-05-09T08:46:46.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:46 smithi133 ceph-mon[29610]: pgmap v882: 97 pgs: 97 active+clean; 618 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-05-09T08:46:46.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:46 smithi188 ceph-mon[34645]: pgmap v882: 97 pgs: 97 active+clean; 618 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 14 MiB/s wr, 718 op/s 2024-05-09T08:46:47.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 106275 30.35 MB/sec execute 62 sec latency 116.340 ms 2024-05-09T08:46:48.362 INFO:tasks.workunit.client.0.smithi133.stdout: 10 106902 30.34 MB/sec execute 63 sec latency 49.685 ms 2024-05-09T08:46:48.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:48 smithi160 ceph-mon[34070]: pgmap v883: 97 pgs: 97 active+clean; 609 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T08:46:48.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:48 smithi133 ceph-mon[29610]: pgmap v883: 97 pgs: 97 active+clean; 609 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T08:46:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:48 smithi188 ceph-mon[34645]: pgmap v883: 97 pgs: 97 active+clean; 609 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 13 MiB/s wr, 678 op/s 2024-05-09T08:46:49.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 107592 30.37 MB/sec execute 64 sec latency 65.100 ms 2024-05-09T08:46:50.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 108288 30.48 MB/sec execute 65 sec latency 69.868 ms 2024-05-09T08:46:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:50 smithi160 ceph-mon[34070]: pgmap v884: 97 pgs: 97 active+clean; 599 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-09T08:46:50.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:50 smithi133 ceph-mon[29610]: pgmap v884: 97 pgs: 97 active+clean; 599 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-09T08:46:50.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:50 smithi188 ceph-mon[34645]: pgmap v884: 97 pgs: 97 active+clean; 599 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 776 op/s 2024-05-09T08:46:51.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 108924 30.59 MB/sec execute 66 sec latency 170.337 ms 2024-05-09T08:46:52.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 109546 30.71 MB/sec execute 67 sec latency 68.892 ms 2024-05-09T08:46:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:52 smithi160 ceph-mon[34070]: pgmap v885: 97 pgs: 97 active+clean; 605 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-09T08:46:52.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:52 smithi133 ceph-mon[29610]: pgmap v885: 97 pgs: 97 active+clean; 605 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-09T08:46:52.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:52 smithi188 ceph-mon[34645]: pgmap v885: 97 pgs: 97 active+clean; 605 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-05-09T08:46:53.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 110099 30.67 MB/sec execute 68 sec latency 81.851 ms 2024-05-09T08:46:54.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 110643 30.64 MB/sec execute 69 sec latency 69.152 ms 2024-05-09T08:46:54.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:54 smithi160 ceph-mon[34070]: pgmap v886: 97 pgs: 97 active+clean; 608 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-09T08:46:54.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:54 smithi133 ceph-mon[29610]: pgmap v886: 97 pgs: 97 active+clean; 608 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-09T08:46:54.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:54 smithi188 ceph-mon[34645]: pgmap v886: 97 pgs: 97 active+clean; 608 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 712 op/s 2024-05-09T08:46:55.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 111264 30.70 MB/sec execute 70 sec latency 68.264 ms 2024-05-09T08:46:56.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 111854 30.65 MB/sec execute 71 sec latency 41.449 ms 2024-05-09T08:46:56.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:56 smithi160 ceph-mon[34070]: pgmap v887: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-05-09T08:46:56.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:56 smithi133 ceph-mon[29610]: pgmap v887: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-05-09T08:46:56.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:56 smithi188 ceph-mon[34645]: pgmap v887: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-05-09T08:46:57.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 112364 30.55 MB/sec execute 72 sec latency 312.390 ms 2024-05-09T08:46:58.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 113111 30.77 MB/sec execute 73 sec latency 104.152 ms 2024-05-09T08:46:58.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:46:58 smithi160 ceph-mon[34070]: pgmap v888: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-05-09T08:46:58.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:46:58 smithi133 ceph-mon[29610]: pgmap v888: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-05-09T08:46:58.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:46:58 smithi188 ceph-mon[34645]: pgmap v888: 97 pgs: 97 active+clean; 621 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 500 op/s 2024-05-09T08:46:59.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 113774 30.94 MB/sec execute 74 sec latency 50.117 ms 2024-05-09T08:47:00.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 114233 30.84 MB/sec execute 75 sec latency 84.693 ms 2024-05-09T08:47:00.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:00 smithi160 ceph-mon[34070]: pgmap v889: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 532 op/s 2024-05-09T08:47:00.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:00 smithi133 ceph-mon[29610]: pgmap v889: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 532 op/s 2024-05-09T08:47:00.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:00 smithi188 ceph-mon[34645]: pgmap v889: 97 pgs: 97 active+clean; 616 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 532 op/s 2024-05-09T08:47:01.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 114720 30.71 MB/sec execute 76 sec latency 73.465 ms 2024-05-09T08:47:02.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 115120 30.63 MB/sec execute 77 sec latency 271.905 ms 2024-05-09T08:47:02.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:02 smithi160 ceph-mon[34070]: pgmap v890: 97 pgs: 97 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-09T08:47:02.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:02 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:47:02.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:02 smithi133 ceph-mon[29610]: pgmap v890: 97 pgs: 97 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-09T08:47:02.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:02 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:47:02.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:02 smithi188 ceph-mon[34645]: pgmap v890: 97 pgs: 97 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 504 op/s 2024-05-09T08:47:02.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:02 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:47:03.363 INFO:tasks.workunit.client.0.smithi133.stdout: 10 115263 30.30 MB/sec execute 78 sec latency 692.933 ms 2024-05-09T08:47:03.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:03 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:03 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:03.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:03 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.364 INFO:tasks.workunit.client.0.smithi133.stdout: 10 115783 30.25 MB/sec execute 79 sec latency 79.794 ms 2024-05-09T08:47:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: pgmap v891: 97 pgs: 97 active+clean; 614 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-05-09T08:47:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:47:04.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:47:04.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: pgmap v891: 97 pgs: 97 active+clean; 614 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:47:04.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: pgmap v891: 97 pgs: 97 active+clean; 614 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.0 MiB/s wr, 444 op/s 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi160 to 176.2M 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi160 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi133 to 176.4M 2024-05-09T08:47:04.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi133 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-09T08:47:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:47:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:47:04.683 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:47:05.364 INFO:tasks.workunit.client.0.smithi133.stdout: 10 116518 30.26 MB/sec execute 80 sec latency 58.277 ms 2024-05-09T08:47:06.364 INFO:tasks.workunit.client.0.smithi133.stdout: 10 117276 30.57 MB/sec execute 81 sec latency 58.870 ms 2024-05-09T08:47:06.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:06 smithi160 ceph-mon[34070]: pgmap v892: 97 pgs: 97 active+clean; 627 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 606 op/s 2024-05-09T08:47:06.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:06 smithi133 ceph-mon[29610]: pgmap v892: 97 pgs: 97 active+clean; 627 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 606 op/s 2024-05-09T08:47:06.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:06 smithi188 ceph-mon[34645]: pgmap v892: 97 pgs: 97 active+clean; 627 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 606 op/s 2024-05-09T08:47:07.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 117890 30.63 MB/sec execute 82 sec latency 57.592 ms 2024-05-09T08:47:08.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 118422 30.49 MB/sec execute 83 sec latency 68.034 ms 2024-05-09T08:47:08.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:08 smithi160 ceph-mon[34070]: pgmap v893: 97 pgs: 97 active+clean; 629 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 586 op/s 2024-05-09T08:47:08.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:08 smithi133 ceph-mon[29610]: pgmap v893: 97 pgs: 97 active+clean; 629 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 586 op/s 2024-05-09T08:47:08.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:08 smithi188 ceph-mon[34645]: pgmap v893: 97 pgs: 97 active+clean; 629 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 11 MiB/s wr, 586 op/s 2024-05-09T08:47:09.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 118990 30.55 MB/sec execute 84 sec latency 78.569 ms 2024-05-09T08:47:10.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 119571 30.54 MB/sec execute 85 sec latency 52.521 ms 2024-05-09T08:47:10.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:10 smithi160 ceph-mon[34070]: pgmap v894: 97 pgs: 97 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:47:10.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:10 smithi133 ceph-mon[29610]: pgmap v894: 97 pgs: 97 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:47:10.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:10 smithi188 ceph-mon[34645]: pgmap v894: 97 pgs: 97 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:47:11.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 120251 30.49 MB/sec execute 86 sec latency 63.156 ms 2024-05-09T08:47:12.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 120958 30.64 MB/sec execute 87 sec latency 105.525 ms 2024-05-09T08:47:12.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:12 smithi160 ceph-mon[34070]: pgmap v895: 97 pgs: 97 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-05-09T08:47:12.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:12 smithi133 ceph-mon[29610]: pgmap v895: 97 pgs: 97 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-05-09T08:47:12.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:12 smithi188 ceph-mon[34645]: pgmap v895: 97 pgs: 97 active+clean; 619 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 683 op/s 2024-05-09T08:47:13.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 121577 30.76 MB/sec execute 88 sec latency 69.954 ms 2024-05-09T08:47:14.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 122169 30.77 MB/sec execute 89 sec latency 65.459 ms 2024-05-09T08:47:14.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:14 smithi160 ceph-mon[34070]: pgmap v896: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 586 op/s 2024-05-09T08:47:14.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:14 smithi133 ceph-mon[29610]: pgmap v896: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 586 op/s 2024-05-09T08:47:14.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:14 smithi188 ceph-mon[34645]: pgmap v896: 97 pgs: 97 active+clean; 623 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 586 op/s 2024-05-09T08:47:15.365 INFO:tasks.workunit.client.0.smithi133.stdout: 10 122588 30.65 MB/sec execute 90 sec latency 117.277 ms 2024-05-09T08:47:16.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 123171 30.68 MB/sec execute 91 sec latency 78.531 ms 2024-05-09T08:47:16.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:16 smithi133 ceph-mon[29610]: pgmap v897: 97 pgs: 97 active+clean; 628 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.1 MiB/s wr, 611 op/s 2024-05-09T08:47:16.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:16 smithi188 ceph-mon[34645]: pgmap v897: 97 pgs: 97 active+clean; 628 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.1 MiB/s wr, 611 op/s 2024-05-09T08:47:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:16 smithi160 ceph-mon[34070]: pgmap v897: 97 pgs: 97 active+clean; 628 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.1 MiB/s wr, 611 op/s 2024-05-09T08:47:17.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 123694 30.62 MB/sec execute 92 sec latency 302.181 ms 2024-05-09T08:47:18.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 124248 30.49 MB/sec execute 93 sec latency 384.343 ms 2024-05-09T08:47:18.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:18 smithi133 ceph-mon[29610]: pgmap v898: 97 pgs: 97 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-05-09T08:47:18.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:18 smithi188 ceph-mon[34645]: pgmap v898: 97 pgs: 97 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-05-09T08:47:18.713 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:18 smithi160 ceph-mon[34070]: pgmap v898: 97 pgs: 97 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.2 MiB/s wr, 473 op/s 2024-05-09T08:47:19.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 124902 30.66 MB/sec execute 94 sec latency 77.317 ms 2024-05-09T08:47:20.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 125595 30.78 MB/sec execute 95 sec latency 53.935 ms 2024-05-09T08:47:20.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:20 smithi133 ceph-mon[29610]: pgmap v899: 97 pgs: 97 active+clean; 638 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-05-09T08:47:20.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:20 smithi188 ceph-mon[34645]: pgmap v899: 97 pgs: 97 active+clean; 638 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-05-09T08:47:20.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:20 smithi160 ceph-mon[34070]: pgmap v899: 97 pgs: 97 active+clean; 638 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-05-09T08:47:21.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 126179 30.73 MB/sec execute 96 sec latency 49.524 ms 2024-05-09T08:47:22.366 INFO:tasks.workunit.client.0.smithi133.stdout: 10 126815 30.79 MB/sec execute 97 sec latency 94.388 ms 2024-05-09T08:47:22.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:22 smithi188 ceph-mon[34645]: pgmap v900: 97 pgs: 97 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-09T08:47:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:22 smithi160 ceph-mon[34070]: pgmap v900: 97 pgs: 97 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-09T08:47:22.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:22 smithi133 ceph-mon[29610]: pgmap v900: 97 pgs: 97 active+clean; 645 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.0 MiB/s wr, 426 op/s 2024-05-09T08:47:23.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 127242 30.66 MB/sec execute 98 sec latency 294.779 ms 2024-05-09T08:47:24.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 127415 30.43 MB/sec execute 99 sec latency 899.975 ms 2024-05-09T08:47:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:24 smithi160 ceph-mon[34070]: pgmap v901: 97 pgs: 97 active+clean; 648 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 422 op/s 2024-05-09T08:47:24.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:24 smithi133 ceph-mon[29610]: pgmap v901: 97 pgs: 97 active+clean; 648 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 422 op/s 2024-05-09T08:47:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:24 smithi188 ceph-mon[34645]: pgmap v901: 97 pgs: 97 active+clean; 648 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 422 op/s 2024-05-09T08:47:25.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 128038 30.49 MB/sec execute 100 sec latency 83.429 ms 2024-05-09T08:47:26.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 128661 30.55 MB/sec execute 101 sec latency 66.810 ms 2024-05-09T08:47:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:26 smithi160 ceph-mon[34070]: pgmap v902: 97 pgs: 97 active+clean; 657 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 584 op/s 2024-05-09T08:47:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:26 smithi133 ceph-mon[29610]: pgmap v902: 97 pgs: 97 active+clean; 657 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 584 op/s 2024-05-09T08:47:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:26 smithi188 ceph-mon[34645]: pgmap v902: 97 pgs: 97 active+clean; 657 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.8 MiB/s wr, 584 op/s 2024-05-09T08:47:27.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 129262 30.63 MB/sec execute 102 sec latency 83.030 ms 2024-05-09T08:47:28.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 129930 30.59 MB/sec execute 103 sec latency 85.767 ms 2024-05-09T08:47:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:28 smithi160 ceph-mon[34070]: pgmap v903: 97 pgs: 97 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-05-09T08:47:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:28 smithi133 ceph-mon[29610]: pgmap v903: 97 pgs: 97 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-05-09T08:47:28.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:28 smithi188 ceph-mon[34645]: pgmap v903: 97 pgs: 97 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-05-09T08:47:29.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 130403 30.60 MB/sec execute 104 sec latency 57.306 ms 2024-05-09T08:47:30.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 130930 30.56 MB/sec execute 105 sec latency 62.775 ms 2024-05-09T08:47:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:30 smithi160 ceph-mon[34070]: pgmap v904: 97 pgs: 97 active+clean; 636 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 677 op/s 2024-05-09T08:47:30.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:30 smithi133 ceph-mon[29610]: pgmap v904: 97 pgs: 97 active+clean; 636 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 677 op/s 2024-05-09T08:47:30.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:30 smithi188 ceph-mon[34645]: pgmap v904: 97 pgs: 97 active+clean; 636 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 677 op/s 2024-05-09T08:47:31.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 131530 30.51 MB/sec execute 106 sec latency 78.848 ms 2024-05-09T08:47:32.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 132293 30.70 MB/sec execute 107 sec latency 64.978 ms 2024-05-09T08:47:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:32 smithi160 ceph-mon[34070]: pgmap v905: 97 pgs: 97 active+clean; 631 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T08:47:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:32 smithi133 ceph-mon[29610]: pgmap v905: 97 pgs: 97 active+clean; 631 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T08:47:32.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:32 smithi188 ceph-mon[34645]: pgmap v905: 97 pgs: 97 active+clean; 631 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 692 op/s 2024-05-09T08:47:33.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 133003 30.71 MB/sec execute 108 sec latency 45.118 ms 2024-05-09T08:47:34.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 133633 30.79 MB/sec execute 109 sec latency 65.026 ms 2024-05-09T08:47:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:34 smithi160 ceph-mon[34070]: pgmap v906: 97 pgs: 97 active+clean; 633 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-05-09T08:47:34.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:34 smithi133 ceph-mon[29610]: pgmap v906: 97 pgs: 97 active+clean; 633 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-05-09T08:47:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:34 smithi188 ceph-mon[34645]: pgmap v906: 97 pgs: 97 active+clean; 633 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-05-09T08:47:35.368 INFO:tasks.workunit.client.0.smithi133.stdout: 10 134228 30.80 MB/sec execute 110 sec latency 54.019 ms 2024-05-09T08:47:36.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 134790 30.75 MB/sec execute 111 sec latency 68.736 ms 2024-05-09T08:47:36.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:36 smithi160 ceph-mon[34070]: pgmap v907: 97 pgs: 97 active+clean; 642 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.4 MiB/s wr, 619 op/s 2024-05-09T08:47:36.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:36 smithi133 ceph-mon[29610]: pgmap v907: 97 pgs: 97 active+clean; 642 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.4 MiB/s wr, 619 op/s 2024-05-09T08:47:36.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:36 smithi188 ceph-mon[34645]: pgmap v907: 97 pgs: 97 active+clean; 642 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 9.4 MiB/s wr, 619 op/s 2024-05-09T08:47:37.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 135378 30.75 MB/sec execute 112 sec latency 239.175 ms 2024-05-09T08:47:38.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 136084 30.85 MB/sec execute 113 sec latency 52.266 ms 2024-05-09T08:47:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:38 smithi160 ceph-mon[34070]: pgmap v908: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 510 op/s 2024-05-09T08:47:38.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:38 smithi133 ceph-mon[29610]: pgmap v908: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 510 op/s 2024-05-09T08:47:38.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:38 smithi188 ceph-mon[34645]: pgmap v908: 97 pgs: 97 active+clean; 649 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 510 op/s 2024-05-09T08:47:39.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 136577 30.81 MB/sec execute 114 sec latency 264.055 ms 2024-05-09T08:47:40.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 137213 30.81 MB/sec execute 115 sec latency 116.615 ms 2024-05-09T08:47:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:40 smithi160 ceph-mon[34070]: pgmap v909: 97 pgs: 97 active+clean; 654 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-05-09T08:47:40.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:40 smithi133 ceph-mon[29610]: pgmap v909: 97 pgs: 97 active+clean; 654 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-05-09T08:47:40.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:40 smithi188 ceph-mon[34645]: pgmap v909: 97 pgs: 97 active+clean; 654 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-05-09T08:47:41.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 137781 30.84 MB/sec execute 116 sec latency 108.874 ms 2024-05-09T08:47:42.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 138389 30.89 MB/sec execute 117 sec latency 79.563 ms 2024-05-09T08:47:42.734 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:42 smithi188 ceph-mon[34645]: pgmap v910: 97 pgs: 97 active+clean; 665 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 424 op/s 2024-05-09T08:47:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:42 smithi160 ceph-mon[34070]: pgmap v910: 97 pgs: 97 active+clean; 665 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 424 op/s 2024-05-09T08:47:42.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:42 smithi133 ceph-mon[29610]: pgmap v910: 97 pgs: 97 active+clean; 665 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.5 MiB/s wr, 424 op/s 2024-05-09T08:47:43.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 139014 30.87 MB/sec execute 118 sec latency 47.948 ms 2024-05-09T08:47:44.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 139427 30.78 MB/sec execute 119 sec latency 302.572 ms 2024-05-09T08:47:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:44 smithi160 ceph-mon[34070]: pgmap v911: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 408 op/s 2024-05-09T08:47:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:44 smithi133 ceph-mon[29610]: pgmap v911: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 408 op/s 2024-05-09T08:47:44.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:44 smithi188 ceph-mon[34645]: pgmap v911: 97 pgs: 97 active+clean; 668 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 408 op/s 2024-05-09T08:47:45.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 139484 30.54 MB/sec execute 120 sec latency 750.945 ms 2024-05-09T08:47:46.369 INFO:tasks.workunit.client.0.smithi133.stdout: 10 140021 30.59 MB/sec execute 121 sec latency 221.956 ms 2024-05-09T08:47:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:46 smithi160 ceph-mon[34070]: pgmap v912: 97 pgs: 97 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.6 MiB/s wr, 548 op/s 2024-05-09T08:47:46.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:46 smithi133 ceph-mon[29610]: pgmap v912: 97 pgs: 97 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.6 MiB/s wr, 548 op/s 2024-05-09T08:47:46.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:46 smithi188 ceph-mon[34645]: pgmap v912: 97 pgs: 97 active+clean; 681 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.6 MiB/s wr, 548 op/s 2024-05-09T08:47:47.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 140695 30.59 MB/sec execute 122 sec latency 76.710 ms 2024-05-09T08:47:48.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 141357 30.64 MB/sec execute 123 sec latency 64.015 ms 2024-05-09T08:47:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:48 smithi160 ceph-mon[34070]: pgmap v913: 97 pgs: 97 active+clean; 673 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 499 op/s 2024-05-09T08:47:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:48 smithi133 ceph-mon[29610]: pgmap v913: 97 pgs: 97 active+clean; 673 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 499 op/s 2024-05-09T08:47:48.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:48 smithi188 ceph-mon[34645]: pgmap v913: 97 pgs: 97 active+clean; 673 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 499 op/s 2024-05-09T08:47:49.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 141982 30.72 MB/sec execute 124 sec latency 84.673 ms 2024-05-09T08:47:50.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 142572 30.72 MB/sec execute 125 sec latency 43.874 ms 2024-05-09T08:47:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:50 smithi160 ceph-mon[34070]: pgmap v914: 97 pgs: 97 active+clean; 670 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-09T08:47:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:50 smithi133 ceph-mon[29610]: pgmap v914: 97 pgs: 97 active+clean; 670 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-09T08:47:50.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:50 smithi188 ceph-mon[34645]: pgmap v914: 97 pgs: 97 active+clean; 670 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-05-09T08:47:51.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 143198 30.67 MB/sec execute 126 sec latency 56.076 ms 2024-05-09T08:47:52.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 143706 30.66 MB/sec execute 127 sec latency 129.147 ms 2024-05-09T08:47:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:52 smithi160 ceph-mon[34070]: pgmap v915: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 688 op/s 2024-05-09T08:47:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:52 smithi133 ceph-mon[29610]: pgmap v915: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 688 op/s 2024-05-09T08:47:52.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:52 smithi188 ceph-mon[34645]: pgmap v915: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 688 op/s 2024-05-09T08:47:53.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 144349 30.75 MB/sec execute 128 sec latency 48.573 ms 2024-05-09T08:47:54.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 144998 30.72 MB/sec execute 129 sec latency 60.164 ms 2024-05-09T08:47:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:54 smithi160 ceph-mon[34070]: pgmap v916: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 627 op/s 2024-05-09T08:47:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:54 smithi133 ceph-mon[29610]: pgmap v916: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 627 op/s 2024-05-09T08:47:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:54 smithi188 ceph-mon[34645]: pgmap v916: 97 pgs: 97 active+clean; 647 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 627 op/s 2024-05-09T08:47:55.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 145606 30.73 MB/sec execute 130 sec latency 80.436 ms 2024-05-09T08:47:56.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 146412 30.85 MB/sec execute 131 sec latency 55.512 ms 2024-05-09T08:47:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:56 smithi160 ceph-mon[34070]: pgmap v917: 97 pgs: 97 active+clean; 658 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:47:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:56 smithi133 ceph-mon[29610]: pgmap v917: 97 pgs: 97 active+clean; 658 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:47:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:56 smithi188 ceph-mon[34645]: pgmap v917: 97 pgs: 97 active+clean; 658 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:47:57.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 147041 30.86 MB/sec execute 132 sec latency 52.270 ms 2024-05-09T08:47:58.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 147656 30.95 MB/sec execute 133 sec latency 45.279 ms 2024-05-09T08:47:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:47:58 smithi160 ceph-mon[34070]: pgmap v918: 97 pgs: 97 active+clean; 661 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-05-09T08:47:58.929 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:47:58 smithi133 ceph-mon[29610]: pgmap v918: 97 pgs: 97 active+clean; 661 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-05-09T08:47:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:47:58 smithi188 ceph-mon[34645]: pgmap v918: 97 pgs: 97 active+clean; 661 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-05-09T08:47:59.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 148173 30.91 MB/sec execute 134 sec latency 110.349 ms 2024-05-09T08:48:00.370 INFO:tasks.workunit.client.0.smithi133.stdout: 10 148809 30.87 MB/sec execute 135 sec latency 53.158 ms 2024-05-09T08:48:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:00 smithi160 ceph-mon[34070]: pgmap v919: 97 pgs: 97 active+clean; 664 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 541 op/s 2024-05-09T08:48:00.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:00 smithi133 ceph-mon[29610]: pgmap v919: 97 pgs: 97 active+clean; 664 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 541 op/s 2024-05-09T08:48:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:00 smithi188 ceph-mon[34645]: pgmap v919: 97 pgs: 97 active+clean; 664 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.1 MiB/s wr, 541 op/s 2024-05-09T08:48:01.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 149366 30.89 MB/sec execute 136 sec latency 85.119 ms 2024-05-09T08:48:02.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 150004 30.92 MB/sec execute 137 sec latency 88.965 ms 2024-05-09T08:48:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:02 smithi160 ceph-mon[34070]: pgmap v920: 97 pgs: 97 active+clean; 669 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-05-09T08:48:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:02 smithi133 ceph-mon[29610]: pgmap v920: 97 pgs: 97 active+clean; 669 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-05-09T08:48:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:02 smithi188 ceph-mon[34645]: pgmap v920: 97 pgs: 97 active+clean; 669 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.6 MiB/s wr, 428 op/s 2024-05-09T08:48:03.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 150650 30.92 MB/sec execute 138 sec latency 95.995 ms 2024-05-09T08:48:04.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 151210 30.96 MB/sec execute 139 sec latency 80.307 ms 2024-05-09T08:48:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: pgmap v921: 97 pgs: 97 active+clean; 677 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-05-09T08:48:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:48:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:04 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: pgmap v921: 97 pgs: 97 active+clean; 677 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:04 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: pgmap v921: 97 pgs: 97 active+clean; 677 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.4 MiB/s wr, 372 op/s 2024-05-09T08:48:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:48:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:04.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:04 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T08:48:05.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 151779 30.95 MB/sec execute 140 sec latency 44.061 ms 2024-05-09T08:48:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:05 smithi160 ceph-mon[34070]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:48:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:05 smithi160 ceph-mon[34070]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:48:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:48:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:48:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:48:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:05 smithi133 ceph-mon[29610]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:48:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:05 smithi133 ceph-mon[29610]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:48:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:48:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:48:05.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:48:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:05 smithi188 ceph-mon[34645]: Adjusting osd_memory_target on smithi188 to 176.2M 2024-05-09T08:48:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:05 smithi188 ceph-mon[34645]: Unable to set osd_memory_target on smithi188 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-09T08:48:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:48:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:48:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:48:06.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 152336 30.97 MB/sec execute 141 sec latency 62.429 ms 2024-05-09T08:48:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:06 smithi160 ceph-mon[34070]: pgmap v922: 97 pgs: 97 active+clean; 685 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-05-09T08:48:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:06 smithi133 ceph-mon[29610]: pgmap v922: 97 pgs: 97 active+clean; 685 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-05-09T08:48:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:06 smithi188 ceph-mon[34645]: pgmap v922: 97 pgs: 97 active+clean; 685 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.0 MiB/s wr, 465 op/s 2024-05-09T08:48:07.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 152849 30.92 MB/sec execute 142 sec latency 59.481 ms 2024-05-09T08:48:08.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 153361 30.87 MB/sec execute 143 sec latency 68.937 ms 2024-05-09T08:48:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:08 smithi160 ceph-mon[34070]: pgmap v923: 97 pgs: 97 active+clean; 683 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-05-09T08:48:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:08 smithi133 ceph-mon[29610]: pgmap v923: 97 pgs: 97 active+clean; 683 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-05-09T08:48:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:08 smithi188 ceph-mon[34645]: pgmap v923: 97 pgs: 97 active+clean; 683 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-05-09T08:48:09.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 154209 30.96 MB/sec execute 144 sec latency 51.343 ms 2024-05-09T08:48:10.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 154884 31.00 MB/sec execute 145 sec latency 98.454 ms 2024-05-09T08:48:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:10 smithi160 ceph-mon[34070]: pgmap v924: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-05-09T08:48:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:10 smithi133 ceph-mon[29610]: pgmap v924: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-05-09T08:48:10.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:10 smithi188 ceph-mon[34645]: pgmap v924: 97 pgs: 97 active+clean; 674 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 353 op/s 2024-05-09T08:48:11.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 155484 31.09 MB/sec execute 146 sec latency 73.359 ms 2024-05-09T08:48:12.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 156022 31.07 MB/sec execute 147 sec latency 52.977 ms 2024-05-09T08:48:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:12 smithi160 ceph-mon[34070]: pgmap v925: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 438 op/s 2024-05-09T08:48:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:12 smithi133 ceph-mon[29610]: pgmap v925: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 438 op/s 2024-05-09T08:48:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:12 smithi188 ceph-mon[34645]: pgmap v925: 97 pgs: 97 active+clean; 671 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 438 op/s 2024-05-09T08:48:13.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 156504 31.03 MB/sec execute 148 sec latency 67.515 ms 2024-05-09T08:48:14.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 157027 30.93 MB/sec execute 149 sec latency 61.174 ms 2024-05-09T08:48:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:14 smithi160 ceph-mon[34070]: pgmap v926: 97 pgs: 97 active+clean; 675 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-05-09T08:48:14.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:14 smithi133 ceph-mon[29610]: pgmap v926: 97 pgs: 97 active+clean; 675 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-05-09T08:48:14.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:14 smithi188 ceph-mon[34645]: pgmap v926: 97 pgs: 97 active+clean; 675 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.0 MiB/s wr, 414 op/s 2024-05-09T08:48:15.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 157581 30.93 MB/sec execute 150 sec latency 93.635 ms 2024-05-09T08:48:16.371 INFO:tasks.workunit.client.0.smithi133.stdout: 10 157921 30.84 MB/sec execute 151 sec latency 427.857 ms 2024-05-09T08:48:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:16 smithi160 ceph-mon[34070]: pgmap v927: 97 pgs: 97 active+clean; 682 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 440 op/s 2024-05-09T08:48:16.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:16 smithi133 ceph-mon[29610]: pgmap v927: 97 pgs: 97 active+clean; 682 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 440 op/s 2024-05-09T08:48:16.933 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:16 smithi188 ceph-mon[34645]: pgmap v927: 97 pgs: 97 active+clean; 682 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 440 op/s 2024-05-09T08:48:17.372 INFO:tasks.workunit.client.0.smithi133.stdout: 10 158405 30.85 MB/sec execute 152 sec latency 295.217 ms 2024-05-09T08:48:18.372 INFO:tasks.workunit.client.0.smithi133.stdout: 10 159056 30.86 MB/sec execute 153 sec latency 349.584 ms 2024-05-09T08:48:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:18 smithi160 ceph-mon[34070]: pgmap v928: 97 pgs: 97 active+clean; 689 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-05-09T08:48:18.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:18 smithi133 ceph-mon[29610]: pgmap v928: 97 pgs: 97 active+clean; 689 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-05-09T08:48:18.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:18 smithi188 ceph-mon[34645]: pgmap v928: 97 pgs: 97 active+clean; 689 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-05-09T08:48:19.372 INFO:tasks.workunit.client.0.smithi133.stdout: 10 159675 30.92 MB/sec execute 154 sec latency 110.535 ms 2024-05-09T08:48:20.372 INFO:tasks.workunit.client.0.smithi133.stdout: 10 160204 30.91 MB/sec execute 155 sec latency 96.565 ms 2024-05-09T08:48:20.917 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:20 smithi160 ceph-mon[34070]: pgmap v929: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-09T08:48:20.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:20 smithi133 ceph-mon[29610]: pgmap v929: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-09T08:48:20.943 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:20 smithi188 ceph-mon[34645]: pgmap v929: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.2 MiB/s wr, 517 op/s 2024-05-09T08:48:21.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 160526 30.84 MB/sec execute 156 sec latency 455.918 ms 2024-05-09T08:48:22.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 160564 30.66 MB/sec execute 157 sec latency 803.107 ms 2024-05-09T08:48:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:22 smithi160 ceph-mon[34070]: pgmap v930: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.4 MiB/s wr, 566 op/s 2024-05-09T08:48:22.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:22 smithi133 ceph-mon[29610]: pgmap v930: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.4 MiB/s wr, 566 op/s 2024-05-09T08:48:22.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:22 smithi188 ceph-mon[34645]: pgmap v930: 97 pgs: 97 active+clean; 694 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 8.4 MiB/s wr, 566 op/s 2024-05-09T08:48:23.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 160875 30.53 MB/sec execute 158 sec latency 553.388 ms 2024-05-09T08:48:24.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 161319 30.44 MB/sec execute 159 sec latency 130.494 ms 2024-05-09T08:48:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:24 smithi160 ceph-mon[34070]: pgmap v931: 97 pgs: 97 active+clean; 725 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:48:24.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:24 smithi133 ceph-mon[29610]: pgmap v931: 97 pgs: 97 active+clean; 725 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:48:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:24 smithi188 ceph-mon[34645]: pgmap v931: 97 pgs: 97 active+clean; 725 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 532 op/s 2024-05-09T08:48:25.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 161933 30.45 MB/sec execute 160 sec latency 84.384 ms 2024-05-09T08:48:26.374 INFO:tasks.workunit.client.0.smithi133.stdout: 10 162684 30.52 MB/sec execute 161 sec latency 67.298 ms 2024-05-09T08:48:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:26 smithi160 ceph-mon[34070]: pgmap v932: 97 pgs: 97 active+clean; 747 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 774 op/s 2024-05-09T08:48:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:26 smithi133 ceph-mon[29610]: pgmap v932: 97 pgs: 97 active+clean; 747 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 774 op/s 2024-05-09T08:48:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:26 smithi188 ceph-mon[34645]: pgmap v932: 97 pgs: 97 active+clean; 747 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 17 MiB/s wr, 774 op/s 2024-05-09T08:48:27.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 163445 30.63 MB/sec execute 162 sec latency 47.669 ms 2024-05-09T08:48:28.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 163961 30.64 MB/sec execute 163 sec latency 99.944 ms 2024-05-09T08:48:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:28 smithi160 ceph-mon[34070]: pgmap v933: 97 pgs: 97 active+clean; 716 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-05-09T08:48:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:28 smithi133 ceph-mon[29610]: pgmap v933: 97 pgs: 97 active+clean; 716 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-05-09T08:48:28.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:28 smithi188 ceph-mon[34645]: pgmap v933: 97 pgs: 97 active+clean; 716 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-05-09T08:48:29.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 164359 30.59 MB/sec execute 164 sec latency 102.041 ms 2024-05-09T08:48:30.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 164876 30.53 MB/sec execute 165 sec latency 89.384 ms 2024-05-09T08:48:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:30 smithi160 ceph-mon[34070]: pgmap v934: 97 pgs: 97 active+clean; 662 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 789 op/s 2024-05-09T08:48:30.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:30 smithi133 ceph-mon[29610]: pgmap v934: 97 pgs: 97 active+clean; 662 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 789 op/s 2024-05-09T08:48:30.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:30 smithi188 ceph-mon[34645]: pgmap v934: 97 pgs: 97 active+clean; 662 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 16 MiB/s wr, 789 op/s 2024-05-09T08:48:31.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 165411 30.51 MB/sec execute 166 sec latency 70.701 ms 2024-05-09T08:48:32.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 165986 30.51 MB/sec execute 167 sec latency 62.648 ms 2024-05-09T08:48:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:32 smithi160 ceph-mon[34070]: pgmap v935: 97 pgs: 97 active+clean; 667 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 739 op/s 2024-05-09T08:48:32.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:32 smithi133 ceph-mon[29610]: pgmap v935: 97 pgs: 97 active+clean; 667 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 739 op/s 2024-05-09T08:48:32.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:32 smithi188 ceph-mon[34645]: pgmap v935: 97 pgs: 97 active+clean; 667 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 15 MiB/s wr, 739 op/s 2024-05-09T08:48:33.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 166594 30.49 MB/sec execute 168 sec latency 101.849 ms 2024-05-09T08:48:34.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 167341 30.56 MB/sec execute 169 sec latency 40.081 ms 2024-05-09T08:48:34.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:34 smithi160 ceph-mon[34070]: pgmap v936: 97 pgs: 97 active+clean; 668 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-05-09T08:48:34.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:34 smithi133 ceph-mon[29610]: pgmap v936: 97 pgs: 97 active+clean; 668 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-05-09T08:48:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:34 smithi188 ceph-mon[34645]: pgmap v936: 97 pgs: 97 active+clean; 668 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 673 op/s 2024-05-09T08:48:35.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 167985 30.65 MB/sec execute 170 sec latency 86.518 ms 2024-05-09T08:48:36.375 INFO:tasks.workunit.client.0.smithi133.stdout: 10 168486 30.64 MB/sec execute 171 sec latency 151.406 ms 2024-05-09T08:48:36.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:36 smithi160 ceph-mon[34070]: pgmap v937: 97 pgs: 97 active+clean; 679 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-09T08:48:36.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:36 smithi133 ceph-mon[29610]: pgmap v937: 97 pgs: 97 active+clean; 679 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-09T08:48:36.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:36 smithi188 ceph-mon[34645]: pgmap v937: 97 pgs: 97 active+clean; 679 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-05-09T08:48:37.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 168912 30.59 MB/sec execute 172 sec latency 241.425 ms 2024-05-09T08:48:38.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 169431 30.54 MB/sec execute 173 sec latency 63.608 ms 2024-05-09T08:48:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:38 smithi160 ceph-mon[34070]: pgmap v938: 97 pgs: 97 active+clean; 680 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-05-09T08:48:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:38 smithi133 ceph-mon[29610]: pgmap v938: 97 pgs: 97 active+clean; 680 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-05-09T08:48:38.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:38 smithi188 ceph-mon[34645]: pgmap v938: 97 pgs: 97 active+clean; 680 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-05-09T08:48:39.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 170133 30.54 MB/sec execute 174 sec latency 47.735 ms 2024-05-09T08:48:40.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 170849 30.60 MB/sec execute 175 sec latency 45.457 ms 2024-05-09T08:48:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:40 smithi160 ceph-mon[34070]: pgmap v939: 97 pgs: 97 active+clean; 687 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-05-09T08:48:40.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:40 smithi133 ceph-mon[29610]: pgmap v939: 97 pgs: 97 active+clean; 687 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-05-09T08:48:40.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:40 smithi188 ceph-mon[34645]: pgmap v939: 97 pgs: 97 active+clean; 687 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 467 op/s 2024-05-09T08:48:41.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 171572 30.67 MB/sec execute 176 sec latency 48.318 ms 2024-05-09T08:48:42.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 172045 30.69 MB/sec execute 177 sec latency 251.537 ms 2024-05-09T08:48:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:42 smithi160 ceph-mon[34070]: pgmap v940: 97 pgs: 97 active+clean; 690 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 404 op/s 2024-05-09T08:48:42.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:42 smithi133 ceph-mon[29610]: pgmap v940: 97 pgs: 97 active+clean; 690 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 404 op/s 2024-05-09T08:48:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:42 smithi188 ceph-mon[34645]: pgmap v940: 97 pgs: 97 active+clean; 690 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 404 op/s 2024-05-09T08:48:43.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 172541 30.62 MB/sec execute 178 sec latency 358.956 ms 2024-05-09T08:48:44.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 173081 30.61 MB/sec execute 179 sec latency 108.802 ms 2024-05-09T08:48:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:44 smithi160 ceph-mon[34070]: pgmap v941: 97 pgs: 97 active+clean; 697 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:48:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:44 smithi133 ceph-mon[29610]: pgmap v941: 97 pgs: 97 active+clean; 697 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:48:44.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:44 smithi188 ceph-mon[34645]: pgmap v941: 97 pgs: 97 active+clean; 697 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:48:45.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 173683 30.61 MB/sec execute 180 sec latency 66.188 ms 2024-05-09T08:48:46.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 174301 30.60 MB/sec execute 181 sec latency 81.967 ms 2024-05-09T08:48:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:46 smithi160 ceph-mon[34070]: pgmap v942: 97 pgs: 97 active+clean; 705 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-05-09T08:48:46.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:46 smithi133 ceph-mon[29610]: pgmap v942: 97 pgs: 97 active+clean; 705 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-05-09T08:48:46.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:46 smithi188 ceph-mon[34645]: pgmap v942: 97 pgs: 97 active+clean; 705 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-05-09T08:48:47.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 174876 30.65 MB/sec execute 182 sec latency 93.086 ms 2024-05-09T08:48:48.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 175482 30.66 MB/sec execute 183 sec latency 150.047 ms 2024-05-09T08:48:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:48 smithi160 ceph-mon[34070]: pgmap v943: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-05-09T08:48:48.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:48 smithi133 ceph-mon[29610]: pgmap v943: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-05-09T08:48:48.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:48 smithi188 ceph-mon[34645]: pgmap v943: 97 pgs: 97 active+clean; 710 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-05-09T08:48:49.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 176100 30.66 MB/sec execute 184 sec latency 64.732 ms 2024-05-09T08:48:50.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 176680 30.72 MB/sec execute 185 sec latency 102.299 ms 2024-05-09T08:48:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:50 smithi160 ceph-mon[34070]: pgmap v944: 97 pgs: 97 active+clean; 713 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-05-09T08:48:50.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:50 smithi133 ceph-mon[29610]: pgmap v944: 97 pgs: 97 active+clean; 713 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-05-09T08:48:50.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:50 smithi188 ceph-mon[34645]: pgmap v944: 97 pgs: 97 active+clean; 713 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 427 op/s 2024-05-09T08:48:51.377 INFO:tasks.workunit.client.0.smithi133.stdout: 10 177196 30.67 MB/sec execute 186 sec latency 92.327 ms 2024-05-09T08:48:52.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 177667 30.60 MB/sec execute 187 sec latency 231.654 ms 2024-05-09T08:48:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:52 smithi160 ceph-mon[34070]: pgmap v945: 97 pgs: 97 active+clean; 716 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-05-09T08:48:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:52 smithi133 ceph-mon[29610]: pgmap v945: 97 pgs: 97 active+clean; 716 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-05-09T08:48:52.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:52 smithi188 ceph-mon[34645]: pgmap v945: 97 pgs: 97 active+clean; 716 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 386 op/s 2024-05-09T08:48:53.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 178215 30.59 MB/sec execute 188 sec latency 281.773 ms 2024-05-09T08:48:54.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 178933 30.66 MB/sec execute 189 sec latency 100.174 ms 2024-05-09T08:48:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:54 smithi160 ceph-mon[34070]: pgmap v946: 97 pgs: 97 active+clean; 718 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 398 op/s 2024-05-09T08:48:54.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:54 smithi133 ceph-mon[29610]: pgmap v946: 97 pgs: 97 active+clean; 718 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 398 op/s 2024-05-09T08:48:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:54 smithi188 ceph-mon[34645]: pgmap v946: 97 pgs: 97 active+clean; 718 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.2 MiB/s wr, 398 op/s 2024-05-09T08:48:55.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 179629 30.71 MB/sec execute 190 sec latency 41.556 ms 2024-05-09T08:48:56.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 180195 30.70 MB/sec execute 191 sec latency 70.602 ms 2024-05-09T08:48:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:56 smithi160 ceph-mon[34070]: pgmap v947: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 487 op/s 2024-05-09T08:48:56.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:56 smithi133 ceph-mon[29610]: pgmap v947: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 487 op/s 2024-05-09T08:48:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:56 smithi188 ceph-mon[34645]: pgmap v947: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 487 op/s 2024-05-09T08:48:57.378 INFO:tasks.workunit.client.0.smithi133.stdout: 10 180625 30.68 MB/sec execute 192 sec latency 96.776 ms 2024-05-09T08:48:58.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 180991 30.61 MB/sec execute 193 sec latency 315.505 ms 2024-05-09T08:48:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:48:58 smithi160 ceph-mon[34070]: pgmap v948: 97 pgs: 97 active+clean; 708 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-05-09T08:48:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:48:58 smithi133 ceph-mon[29610]: pgmap v948: 97 pgs: 97 active+clean; 708 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-05-09T08:48:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:48:58 smithi188 ceph-mon[34645]: pgmap v948: 97 pgs: 97 active+clean; 708 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.4 MiB/s wr, 418 op/s 2024-05-09T08:48:59.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 181670 30.61 MB/sec execute 194 sec latency 82.758 ms 2024-05-09T08:49:00.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 182316 30.65 MB/sec execute 195 sec latency 57.743 ms 2024-05-09T08:49:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:00 smithi160 ceph-mon[34070]: pgmap v949: 97 pgs: 97 active+clean; 709 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-05-09T08:49:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:00 smithi133 ceph-mon[29610]: pgmap v949: 97 pgs: 97 active+clean; 709 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-05-09T08:49:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:00 smithi188 ceph-mon[34645]: pgmap v949: 97 pgs: 97 active+clean; 709 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-05-09T08:49:01.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 182908 30.68 MB/sec execute 196 sec latency 70.409 ms 2024-05-09T08:49:02.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 183476 30.67 MB/sec execute 197 sec latency 71.658 ms 2024-05-09T08:49:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:02 smithi160 ceph-mon[34070]: pgmap v950: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 505 op/s 2024-05-09T08:49:02.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:02 smithi133 ceph-mon[29610]: pgmap v950: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 505 op/s 2024-05-09T08:49:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:02 smithi188 ceph-mon[34645]: pgmap v950: 97 pgs: 97 active+clean; 716 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.0 MiB/s wr, 505 op/s 2024-05-09T08:49:03.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 183855 30.63 MB/sec execute 198 sec latency 238.434 ms 2024-05-09T08:49:04.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 184428 30.60 MB/sec execute 199 sec latency 84.091 ms 2024-05-09T08:49:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:04 smithi160 ceph-mon[34070]: pgmap v951: 97 pgs: 97 active+clean; 723 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.9 MiB/s wr, 571 op/s 2024-05-09T08:49:04.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:04 smithi133 ceph-mon[29610]: pgmap v951: 97 pgs: 97 active+clean; 723 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.9 MiB/s wr, 571 op/s 2024-05-09T08:49:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:04 smithi188 ceph-mon[34645]: pgmap v951: 97 pgs: 97 active+clean; 723 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.9 MiB/s wr, 571 op/s 2024-05-09T08:49:05.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 184985 30.61 MB/sec execute 200 sec latency 71.478 ms 2024-05-09T08:49:05.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:05 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:49:05.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:05 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:49:05.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:05 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:49:06.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 185654 30.62 MB/sec execute 201 sec latency 74.752 ms 2024-05-09T08:49:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:06 smithi133 ceph-mon[29610]: pgmap v952: 97 pgs: 97 active+clean; 725 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 601 op/s 2024-05-09T08:49:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:06 smithi188 ceph-mon[34645]: pgmap v952: 97 pgs: 97 active+clean; 725 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 601 op/s 2024-05-09T08:49:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:06 smithi160 ceph-mon[34070]: pgmap v952: 97 pgs: 97 active+clean; 725 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.7 MiB/s wr, 601 op/s 2024-05-09T08:49:07.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 186307 30.64 MB/sec execute 202 sec latency 63.625 ms 2024-05-09T08:49:07.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:49:07.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:49:07.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:49:07.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:49:07.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:49:07.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:49:08.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:49:08.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:49:08.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:49:08.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 186743 30.62 MB/sec execute 203 sec latency 222.193 ms 2024-05-09T08:49:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:08 smithi188 ceph-mon[34645]: pgmap v953: 97 pgs: 97 active+clean; 728 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.2 MiB/s wr, 495 op/s 2024-05-09T08:49:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:08 smithi160 ceph-mon[34070]: pgmap v953: 97 pgs: 97 active+clean; 728 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.2 MiB/s wr, 495 op/s 2024-05-09T08:49:09.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:08 smithi133 ceph-mon[29610]: pgmap v953: 97 pgs: 97 active+clean; 728 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.2 MiB/s wr, 495 op/s 2024-05-09T08:49:09.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 187332 30.65 MB/sec execute 204 sec latency 113.023 ms 2024-05-09T08:49:10.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 187891 30.64 MB/sec execute 205 sec latency 63.722 ms 2024-05-09T08:49:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:10 smithi160 ceph-mon[34070]: pgmap v954: 97 pgs: 97 active+clean; 735 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 590 op/s 2024-05-09T08:49:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:10 smithi188 ceph-mon[34645]: pgmap v954: 97 pgs: 97 active+clean; 735 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 590 op/s 2024-05-09T08:49:11.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 188417 30.62 MB/sec execute 206 sec latency 219.154 ms 2024-05-09T08:49:11.967 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:10 smithi133 ceph-mon[29610]: pgmap v954: 97 pgs: 97 active+clean; 735 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 590 op/s 2024-05-09T08:49:12.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 188864 30.59 MB/sec execute 207 sec latency 275.475 ms 2024-05-09T08:49:12.983 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:12 smithi188 ceph-mon[34645]: pgmap v955: 97 pgs: 97 active+clean; 736 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 583 op/s 2024-05-09T08:49:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:12 smithi160 ceph-mon[34070]: pgmap v955: 97 pgs: 97 active+clean; 736 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 583 op/s 2024-05-09T08:49:13.379 INFO:tasks.workunit.client.0.smithi133.stdout: 10 188877 30.44 MB/sec execute 208 sec latency 803.685 ms 2024-05-09T08:49:13.815 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:12 smithi133 ceph-mon[29610]: pgmap v955: 97 pgs: 97 active+clean; 736 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.5 MiB/s wr, 583 op/s 2024-05-09T08:49:14.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 188952 30.31 MB/sec execute 209 sec latency 1111.150 ms 2024-05-09T08:49:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:14 smithi160 ceph-mon[34070]: pgmap v956: 97 pgs: 97 active+clean; 740 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.7 MiB/s wr, 515 op/s 2024-05-09T08:49:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:14 smithi133 ceph-mon[29610]: pgmap v956: 97 pgs: 97 active+clean; 740 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.7 MiB/s wr, 515 op/s 2024-05-09T08:49:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:14 smithi188 ceph-mon[34645]: pgmap v956: 97 pgs: 97 active+clean; 740 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.7 MiB/s wr, 515 op/s 2024-05-09T08:49:15.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 189440 30.27 MB/sec execute 210 sec latency 199.824 ms 2024-05-09T08:49:16.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 189917 30.26 MB/sec execute 211 sec latency 224.302 ms 2024-05-09T08:49:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:16 smithi160 ceph-mon[34070]: pgmap v957: 97 pgs: 97 active+clean; 789 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 667 op/s 2024-05-09T08:49:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:16 smithi188 ceph-mon[34645]: pgmap v957: 97 pgs: 97 active+clean; 789 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 667 op/s 2024-05-09T08:49:17.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 190676 30.35 MB/sec execute 212 sec latency 133.161 ms 2024-05-09T08:49:17.437 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:16 smithi133 ceph-mon[29610]: pgmap v957: 97 pgs: 97 active+clean; 789 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 667 op/s 2024-05-09T08:49:18.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 191183 30.31 MB/sec execute 213 sec latency 74.497 ms 2024-05-09T08:49:18.963 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:18 smithi160 ceph-mon[34070]: pgmap v958: 97 pgs: 97 active+clean; 791 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 730 op/s 2024-05-09T08:49:18.988 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:18 smithi133 ceph-mon[29610]: pgmap v958: 97 pgs: 97 active+clean; 791 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 730 op/s 2024-05-09T08:49:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:18 smithi188 ceph-mon[34645]: pgmap v958: 97 pgs: 97 active+clean; 791 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 18 MiB/s wr, 730 op/s 2024-05-09T08:49:19.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 191866 30.33 MB/sec execute 214 sec latency 52.121 ms 2024-05-09T08:49:20.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 192510 30.37 MB/sec execute 215 sec latency 124.178 ms 2024-05-09T08:49:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:20 smithi160 ceph-mon[34070]: pgmap v959: 97 pgs: 97 active+clean; 732 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 839 op/s 2024-05-09T08:49:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:20 smithi133 ceph-mon[29610]: pgmap v959: 97 pgs: 97 active+clean; 732 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 839 op/s 2024-05-09T08:49:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:20 smithi188 ceph-mon[34645]: pgmap v959: 97 pgs: 97 active+clean; 732 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 20 MiB/s wr, 839 op/s 2024-05-09T08:49:21.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 193145 30.38 MB/sec execute 216 sec latency 52.362 ms 2024-05-09T08:49:22.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 193706 30.37 MB/sec execute 217 sec latency 83.349 ms 2024-05-09T08:49:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:22 smithi160 ceph-mon[34070]: pgmap v960: 97 pgs: 97 active+clean; 711 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-09T08:49:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:22 smithi133 ceph-mon[29610]: pgmap v960: 97 pgs: 97 active+clean; 711 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-09T08:49:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:22 smithi188 ceph-mon[34645]: pgmap v960: 97 pgs: 97 active+clean; 711 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 19 MiB/s wr, 874 op/s 2024-05-09T08:49:23.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 194391 30.44 MB/sec execute 218 sec latency 72.258 ms 2024-05-09T08:49:24.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 194973 30.39 MB/sec execute 219 sec latency 94.254 ms 2024-05-09T08:49:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:24 smithi160 ceph-mon[34070]: pgmap v961: 97 pgs: 97 active+clean; 710 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 843 op/s 2024-05-09T08:49:25.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:24 smithi133 ceph-mon[29610]: pgmap v961: 97 pgs: 97 active+clean; 710 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 843 op/s 2024-05-09T08:49:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:24 smithi188 ceph-mon[34645]: pgmap v961: 97 pgs: 97 active+clean; 710 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 18 MiB/s wr, 843 op/s 2024-05-09T08:49:25.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 195661 30.43 MB/sec execute 220 sec latency 75.853 ms 2024-05-09T08:49:26.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 196293 30.47 MB/sec execute 221 sec latency 91.129 ms 2024-05-09T08:49:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:26 smithi160 ceph-mon[34070]: pgmap v962: 97 pgs: 97 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 904 op/s 2024-05-09T08:49:27.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:26 smithi133 ceph-mon[29610]: pgmap v962: 97 pgs: 97 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 904 op/s 2024-05-09T08:49:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:26 smithi188 ceph-mon[34645]: pgmap v962: 97 pgs: 97 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 18 MiB/s wr, 904 op/s 2024-05-09T08:49:27.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 196872 30.45 MB/sec execute 222 sec latency 56.297 ms 2024-05-09T08:49:28.380 INFO:tasks.workunit.client.0.smithi133.stdout: 10 197510 30.46 MB/sec execute 223 sec latency 58.567 ms 2024-05-09T08:49:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:28 smithi160 ceph-mon[34070]: pgmap v963: 97 pgs: 97 active+clean; 719 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 680 op/s 2024-05-09T08:49:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:28 smithi133 ceph-mon[29610]: pgmap v963: 97 pgs: 97 active+clean; 719 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 680 op/s 2024-05-09T08:49:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:28 smithi188 ceph-mon[34645]: pgmap v963: 97 pgs: 97 active+clean; 719 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.9 MiB/s wr, 680 op/s 2024-05-09T08:49:29.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 198101 30.50 MB/sec execute 224 sec latency 59.028 ms 2024-05-09T08:49:30.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 198573 30.47 MB/sec execute 225 sec latency 92.378 ms 2024-05-09T08:49:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:30 smithi160 ceph-mon[34070]: pgmap v964: 97 pgs: 97 active+clean; 721 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-05-09T08:49:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:30 smithi133 ceph-mon[29610]: pgmap v964: 97 pgs: 97 active+clean; 721 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-05-09T08:49:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:30 smithi188 ceph-mon[34645]: pgmap v964: 97 pgs: 97 active+clean; 721 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 558 op/s 2024-05-09T08:49:31.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 199159 30.44 MB/sec execute 226 sec latency 92.672 ms 2024-05-09T08:49:32.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 199783 30.46 MB/sec execute 227 sec latency 81.499 ms 2024-05-09T08:49:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:32 smithi160 ceph-mon[34070]: pgmap v965: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-05-09T08:49:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:32 smithi133 ceph-mon[29610]: pgmap v965: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-05-09T08:49:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:32 smithi188 ceph-mon[34645]: pgmap v965: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 462 op/s 2024-05-09T08:49:33.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 200414 30.50 MB/sec execute 228 sec latency 88.322 ms 2024-05-09T08:49:34.381 INFO:tasks.workunit.client.0.smithi133.stdout: 10 201181 30.54 MB/sec execute 229 sec latency 44.402 ms 2024-05-09T08:49:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:34 smithi160 ceph-mon[34070]: pgmap v966: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 376 op/s 2024-05-09T08:49:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:34 smithi133 ceph-mon[29610]: pgmap v966: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 376 op/s 2024-05-09T08:49:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:34 smithi188 ceph-mon[34645]: pgmap v966: 97 pgs: 97 active+clean; 729 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 376 op/s 2024-05-09T08:49:35.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 201715 30.56 MB/sec execute 230 sec latency 79.355 ms 2024-05-09T08:49:36.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 202270 30.56 MB/sec execute 231 sec latency 67.751 ms 2024-05-09T08:49:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:36 smithi160 ceph-mon[34070]: pgmap v967: 97 pgs: 97 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:49:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:36 smithi133 ceph-mon[29610]: pgmap v967: 97 pgs: 97 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:49:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:36 smithi188 ceph-mon[34645]: pgmap v967: 97 pgs: 97 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2024-05-09T08:49:37.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 202640 30.49 MB/sec execute 232 sec latency 194.830 ms 2024-05-09T08:49:38.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 203144 30.45 MB/sec execute 233 sec latency 73.616 ms 2024-05-09T08:49:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:38 smithi160 ceph-mon[34070]: pgmap v968: 97 pgs: 97 active+clean; 723 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-05-09T08:49:39.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:38 smithi133 ceph-mon[29610]: pgmap v968: 97 pgs: 97 active+clean; 723 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-05-09T08:49:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:38 smithi188 ceph-mon[34645]: pgmap v968: 97 pgs: 97 active+clean; 723 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.3 MiB/s wr, 390 op/s 2024-05-09T08:49:39.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 203826 30.49 MB/sec execute 234 sec latency 50.752 ms 2024-05-09T08:49:40.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 204551 30.53 MB/sec execute 235 sec latency 49.322 ms 2024-05-09T08:49:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:40 smithi160 ceph-mon[34070]: pgmap v969: 97 pgs: 97 active+clean; 727 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.7 MiB/s wr, 423 op/s 2024-05-09T08:49:41.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:40 smithi133 ceph-mon[29610]: pgmap v969: 97 pgs: 97 active+clean; 727 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.7 MiB/s wr, 423 op/s 2024-05-09T08:49:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:40 smithi188 ceph-mon[34645]: pgmap v969: 97 pgs: 97 active+clean; 727 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.7 MiB/s wr, 423 op/s 2024-05-09T08:49:41.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 205327 30.58 MB/sec execute 236 sec latency 111.487 ms 2024-05-09T08:49:42.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 205596 30.53 MB/sec execute 237 sec latency 430.561 ms 2024-05-09T08:49:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:42 smithi160 ceph-mon[34070]: pgmap v970: 97 pgs: 97 active+clean; 733 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 518 op/s 2024-05-09T08:49:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:42 smithi133 ceph-mon[29610]: pgmap v970: 97 pgs: 97 active+clean; 733 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 518 op/s 2024-05-09T08:49:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:42 smithi188 ceph-mon[34645]: pgmap v970: 97 pgs: 97 active+clean; 733 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 518 op/s 2024-05-09T08:49:43.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 205751 30.46 MB/sec execute 238 sec latency 807.856 ms 2024-05-09T08:49:44.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 206111 30.41 MB/sec execute 239 sec latency 103.725 ms 2024-05-09T08:49:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:44 smithi160 ceph-mon[34070]: pgmap v971: 97 pgs: 97 active+clean; 739 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-05-09T08:49:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:44 smithi133 ceph-mon[29610]: pgmap v971: 97 pgs: 97 active+clean; 739 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-05-09T08:49:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:44 smithi188 ceph-mon[34645]: pgmap v971: 97 pgs: 97 active+clean; 739 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-05-09T08:49:45.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 206620 30.38 MB/sec execute 240 sec latency 77.660 ms 2024-05-09T08:49:46.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 207164 30.35 MB/sec execute 241 sec latency 73.942 ms 2024-05-09T08:49:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:46 smithi160 ceph-mon[34070]: pgmap v972: 97 pgs: 97 active+clean; 755 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-05-09T08:49:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:46 smithi133 ceph-mon[29610]: pgmap v972: 97 pgs: 97 active+clean; 755 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-05-09T08:49:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:46 smithi188 ceph-mon[34645]: pgmap v972: 97 pgs: 97 active+clean; 755 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 12 MiB/s wr, 648 op/s 2024-05-09T08:49:47.382 INFO:tasks.workunit.client.0.smithi133.stdout: 10 207837 30.35 MB/sec execute 242 sec latency 55.846 ms 2024-05-09T08:49:48.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 208628 30.41 MB/sec execute 243 sec latency 53.486 ms 2024-05-09T08:49:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:48 smithi160 ceph-mon[34070]: pgmap v973: 97 pgs: 97 active+clean; 756 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-05-09T08:49:49.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:48 smithi133 ceph-mon[29610]: pgmap v973: 97 pgs: 97 active+clean; 756 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-05-09T08:49:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:48 smithi188 ceph-mon[34645]: pgmap v973: 97 pgs: 97 active+clean; 756 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 12 MiB/s wr, 639 op/s 2024-05-09T08:49:49.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 209395 30.51 MB/sec execute 244 sec latency 59.622 ms 2024-05-09T08:49:50.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 209914 30.50 MB/sec execute 245 sec latency 82.326 ms 2024-05-09T08:49:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:50 smithi160 ceph-mon[34070]: pgmap v974: 97 pgs: 97 active+clean; 734 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-05-09T08:49:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:50 smithi133 ceph-mon[29610]: pgmap v974: 97 pgs: 97 active+clean; 734 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-05-09T08:49:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:50 smithi188 ceph-mon[34645]: pgmap v974: 97 pgs: 97 active+clean; 734 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 631 op/s 2024-05-09T08:49:51.387 INFO:tasks.workunit.client.0.smithi133.stdout: 10 210429 30.47 MB/sec execute 246 sec latency 87.019 ms 2024-05-09T08:49:52.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 210896 30.45 MB/sec execute 247 sec latency 69.568 ms 2024-05-09T08:49:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:52 smithi160 ceph-mon[34070]: pgmap v975: 97 pgs: 97 active+clean; 741 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-05-09T08:49:53.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:52 smithi133 ceph-mon[29610]: pgmap v975: 97 pgs: 97 active+clean; 741 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-05-09T08:49:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:52 smithi188 ceph-mon[34645]: pgmap v975: 97 pgs: 97 active+clean; 741 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-05-09T08:49:53.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 211421 30.42 MB/sec execute 248 sec latency 86.036 ms 2024-05-09T08:49:54.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 212097 30.42 MB/sec execute 249 sec latency 40.215 ms 2024-05-09T08:49:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:54 smithi160 ceph-mon[34070]: pgmap v976: 97 pgs: 97 active+clean; 746 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-05-09T08:49:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:54 smithi133 ceph-mon[29610]: pgmap v976: 97 pgs: 97 active+clean; 746 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-05-09T08:49:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:54 smithi188 ceph-mon[34645]: pgmap v976: 97 pgs: 97 active+clean; 746 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 10 MiB/s wr, 585 op/s 2024-05-09T08:49:55.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 212834 30.46 MB/sec execute 250 sec latency 41.381 ms 2024-05-09T08:49:56.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 213477 30.54 MB/sec execute 251 sec latency 205.395 ms 2024-05-09T08:49:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:56 smithi160 ceph-mon[34070]: pgmap v977: 97 pgs: 97 active+clean; 736 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 632 op/s 2024-05-09T08:49:57.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:56 smithi133 ceph-mon[29610]: pgmap v977: 97 pgs: 97 active+clean; 736 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 632 op/s 2024-05-09T08:49:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:56 smithi188 ceph-mon[34645]: pgmap v977: 97 pgs: 97 active+clean; 736 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.9 MiB/s wr, 632 op/s 2024-05-09T08:49:57.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 214025 30.52 MB/sec execute 252 sec latency 77.299 ms 2024-05-09T08:49:58.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 214252 30.45 MB/sec execute 253 sec latency 344.957 ms 2024-05-09T08:49:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:49:58 smithi160 ceph-mon[34070]: pgmap v978: 97 pgs: 97 active+clean; 728 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-09T08:49:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:49:58 smithi133 ceph-mon[29610]: pgmap v978: 97 pgs: 97 active+clean; 728 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-09T08:49:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:49:58 smithi188 ceph-mon[34645]: pgmap v978: 97 pgs: 97 active+clean; 728 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-05-09T08:49:59.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 214627 30.42 MB/sec execute 254 sec latency 223.626 ms 2024-05-09T08:50:00.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 215080 30.38 MB/sec execute 255 sec latency 93.333 ms 2024-05-09T08:50:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:00 smithi160 ceph-mon[34070]: pgmap v979: 97 pgs: 97 active+clean; 743 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 428 op/s 2024-05-09T08:50:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:00 smithi160 ceph-mon[34070]: overall HEALTH_OK 2024-05-09T08:50:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:00 smithi133 ceph-mon[29610]: pgmap v979: 97 pgs: 97 active+clean; 743 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 428 op/s 2024-05-09T08:50:01.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:00 smithi133 ceph-mon[29610]: overall HEALTH_OK 2024-05-09T08:50:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:00 smithi188 ceph-mon[34645]: pgmap v979: 97 pgs: 97 active+clean; 743 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 428 op/s 2024-05-09T08:50:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:00 smithi188 ceph-mon[34645]: overall HEALTH_OK 2024-05-09T08:50:01.383 INFO:tasks.workunit.client.0.smithi133.stdout: 10 215702 30.36 MB/sec execute 256 sec latency 59.805 ms 2024-05-09T08:50:02.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 216289 30.35 MB/sec execute 257 sec latency 65.337 ms 2024-05-09T08:50:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:02 smithi160 ceph-mon[34070]: pgmap v980: 97 pgs: 97 active+clean; 747 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 484 op/s 2024-05-09T08:50:03.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:02 smithi133 ceph-mon[29610]: pgmap v980: 97 pgs: 97 active+clean; 747 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 484 op/s 2024-05-09T08:50:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:02 smithi188 ceph-mon[34645]: pgmap v980: 97 pgs: 97 active+clean; 747 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.6 MiB/s wr, 484 op/s 2024-05-09T08:50:03.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 216949 30.40 MB/sec execute 258 sec latency 66.683 ms 2024-05-09T08:50:04.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 217163 30.34 MB/sec execute 259 sec latency 475.207 ms 2024-05-09T08:50:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:04 smithi160 ceph-mon[34070]: pgmap v981: 97 pgs: 97 active+clean; 750 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.2 MiB/s wr, 461 op/s 2024-05-09T08:50:05.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:04 smithi133 ceph-mon[29610]: pgmap v981: 97 pgs: 97 active+clean; 750 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.2 MiB/s wr, 461 op/s 2024-05-09T08:50:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:04 smithi188 ceph-mon[34645]: pgmap v981: 97 pgs: 97 active+clean; 750 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.2 MiB/s wr, 461 op/s 2024-05-09T08:50:05.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 217726 30.36 MB/sec execute 260 sec latency 125.797 ms 2024-05-09T08:50:06.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 218198 30.32 MB/sec execute 261 sec latency 120.148 ms 2024-05-09T08:50:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:06 smithi160 ceph-mon[34070]: pgmap v982: 97 pgs: 97 active+clean; 766 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-09T08:50:07.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:06 smithi133 ceph-mon[29610]: pgmap v982: 97 pgs: 97 active+clean; 766 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-09T08:50:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:06 smithi188 ceph-mon[34645]: pgmap v982: 97 pgs: 97 active+clean; 766 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 10 MiB/s wr, 597 op/s 2024-05-09T08:50:07.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 218758 30.32 MB/sec execute 262 sec latency 99.056 ms 2024-05-09T08:50:08.101 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:07 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:50:08.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:07 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:50:08.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:07 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:50:08.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 219279 30.32 MB/sec execute 263 sec latency 92.853 ms 2024-05-09T08:50:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:08 smithi160 ceph-mon[34070]: pgmap v983: 97 pgs: 97 active+clean; 766 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-05-09T08:50:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:08 smithi133 ceph-mon[29610]: pgmap v983: 97 pgs: 97 active+clean; 766 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-05-09T08:50:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:08 smithi188 ceph-mon[34645]: pgmap v983: 97 pgs: 97 active+clean; 766 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 574 op/s 2024-05-09T08:50:09.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 219934 30.31 MB/sec execute 264 sec latency 47.198 ms 2024-05-09T08:50:10.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 220522 30.32 MB/sec execute 265 sec latency 82.421 ms 2024-05-09T08:50:11.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 220709 30.24 MB/sec execute 266 sec latency 288.585 ms 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: pgmap v984: 97 pgs: 97 active+clean; 754 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:50:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: pgmap v984: 97 pgs: 97 active+clean; 754 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:50:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: pgmap v984: 97 pgs: 97 active+clean; 754 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:50:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:50:12.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 221254 30.26 MB/sec execute 267 sec latency 126.341 ms 2024-05-09T08:50:13.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:12 smithi188 ceph-mon[34645]: pgmap v985: 97 pgs: 97 active+clean; 749 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 637 op/s 2024-05-09T08:50:13.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 221868 30.26 MB/sec execute 268 sec latency 60.089 ms 2024-05-09T08:50:13.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:12 smithi160 ceph-mon[34070]: pgmap v985: 97 pgs: 97 active+clean; 749 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 637 op/s 2024-05-09T08:50:13.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:12 smithi133 ceph-mon[29610]: pgmap v985: 97 pgs: 97 active+clean; 749 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 637 op/s 2024-05-09T08:50:14.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 222368 30.25 MB/sec execute 269 sec latency 71.885 ms 2024-05-09T08:50:15.384 INFO:tasks.workunit.client.0.smithi133.stdout: 10 222825 30.24 MB/sec execute 270 sec latency 101.333 ms 2024-05-09T08:50:15.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:14 smithi160 ceph-mon[34070]: pgmap v986: 97 pgs: 97 active+clean; 755 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 572 op/s 2024-05-09T08:50:15.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:14 smithi133 ceph-mon[29610]: pgmap v986: 97 pgs: 97 active+clean; 755 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 572 op/s 2024-05-09T08:50:15.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:14 smithi188 ceph-mon[34645]: pgmap v986: 97 pgs: 97 active+clean; 755 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 9.6 MiB/s wr, 572 op/s 2024-05-09T08:50:16.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 223413 30.23 MB/sec execute 271 sec latency 94.144 ms 2024-05-09T08:50:17.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 224119 30.24 MB/sec execute 272 sec latency 80.443 ms 2024-05-09T08:50:17.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:16 smithi160 ceph-mon[34070]: pgmap v987: 97 pgs: 97 active+clean; 745 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:50:17.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:16 smithi133 ceph-mon[29610]: pgmap v987: 97 pgs: 97 active+clean; 745 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:50:17.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:16 smithi188 ceph-mon[34645]: pgmap v987: 97 pgs: 97 active+clean; 745 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:50:18.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 224758 30.28 MB/sec execute 273 sec latency 60.235 ms 2024-05-09T08:50:19.213 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:18 smithi160 ceph-mon[34070]: pgmap v988: 97 pgs: 97 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.6 MiB/s wr, 602 op/s 2024-05-09T08:50:19.229 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:18 smithi133 ceph-mon[29610]: pgmap v988: 97 pgs: 97 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.6 MiB/s wr, 602 op/s 2024-05-09T08:50:19.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 225370 30.32 MB/sec execute 274 sec latency 51.353 ms 2024-05-09T08:50:19.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:18 smithi188 ceph-mon[34645]: pgmap v988: 97 pgs: 97 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.6 MiB/s wr, 602 op/s 2024-05-09T08:50:20.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 225494 30.25 MB/sec execute 275 sec latency 331.560 ms 2024-05-09T08:50:21.385 INFO:tasks.workunit.client.0.smithi133.stdout: 10 225912 30.18 MB/sec execute 276 sec latency 122.614 ms 2024-05-09T08:50:21.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:20 smithi160 ceph-mon[34070]: pgmap v989: 97 pgs: 97 active+clean; 735 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 630 op/s 2024-05-09T08:50:21.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:20 smithi133 ceph-mon[29610]: pgmap v989: 97 pgs: 97 active+clean; 735 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 630 op/s 2024-05-09T08:50:21.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:20 smithi188 ceph-mon[34645]: pgmap v989: 97 pgs: 97 active+clean; 735 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 630 op/s 2024-05-09T08:50:22.386 INFO:tasks.workunit.client.0.smithi133.stdout: 10 226420 30.16 MB/sec execute 277 sec latency 70.071 ms 2024-05-09T08:50:23.386 INFO:tasks.workunit.client.0.smithi133.stdout: 10 226973 30.15 MB/sec execute 278 sec latency 100.435 ms 2024-05-09T08:50:23.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:22 smithi160 ceph-mon[34070]: pgmap v990: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 559 op/s 2024-05-09T08:50:23.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:22 smithi133 ceph-mon[29610]: pgmap v990: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 559 op/s 2024-05-09T08:50:23.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:22 smithi188 ceph-mon[34645]: pgmap v990: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 8.8 MiB/s wr, 559 op/s 2024-05-09T08:50:24.386 INFO:tasks.workunit.client.0.smithi133.stdout: 10 227596 30.17 MB/sec execute 279 sec latency 109.577 ms 2024-05-09T08:50:25.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 228280 30.19 MB/sec execute 280 sec latency 101.017 ms 2024-05-09T08:50:25.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:24 smithi160 ceph-mon[34070]: pgmap v991: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.1 MiB/s wr, 557 op/s 2024-05-09T08:50:25.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:24 smithi133 ceph-mon[29610]: pgmap v991: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.1 MiB/s wr, 557 op/s 2024-05-09T08:50:25.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:24 smithi188 ceph-mon[34645]: pgmap v991: 97 pgs: 97 active+clean; 746 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.1 MiB/s wr, 557 op/s 2024-05-09T08:50:26.387 INFO:tasks.workunit.client.0.smithi133.stdout: 10 228642 30.16 MB/sec execute 281 sec latency 251.768 ms 2024-05-09T08:50:27.388 INFO:tasks.workunit.client.0.smithi133.stdout: 10 229160 30.17 MB/sec execute 282 sec latency 110.472 ms 2024-05-09T08:50:27.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:26 smithi160 ceph-mon[34070]: pgmap v992: 97 pgs: 97 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-09T08:50:27.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:26 smithi133 ceph-mon[29610]: pgmap v992: 97 pgs: 97 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-09T08:50:27.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:26 smithi188 ceph-mon[34645]: pgmap v992: 97 pgs: 97 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 707 op/s 2024-05-09T08:50:28.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 229698 30.16 MB/sec execute 283 sec latency 72.138 ms 2024-05-09T08:50:29.388 INFO:tasks.workunit.client.0.smithi133.stdout: 10 230282 30.15 MB/sec execute 284 sec latency 69.591 ms 2024-05-09T08:50:29.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:28 smithi160 ceph-mon[34070]: pgmap v993: 97 pgs: 97 active+clean; 750 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-05-09T08:50:29.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:28 smithi133 ceph-mon[29610]: pgmap v993: 97 pgs: 97 active+clean; 750 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-05-09T08:50:29.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:28 smithi188 ceph-mon[34645]: pgmap v993: 97 pgs: 97 active+clean; 750 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-05-09T08:50:30.388 INFO:tasks.workunit.client.0.smithi133.stdout: 10 230827 30.14 MB/sec execute 285 sec latency 60.859 ms 2024-05-09T08:50:31.388 INFO:tasks.workunit.client.0.smithi133.stdout: 10 231504 30.16 MB/sec execute 286 sec latency 81.976 ms 2024-05-09T08:50:31.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:30 smithi160 ceph-mon[34070]: pgmap v994: 97 pgs: 97 active+clean; 756 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 579 op/s 2024-05-09T08:50:31.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:30 smithi133 ceph-mon[29610]: pgmap v994: 97 pgs: 97 active+clean; 756 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 579 op/s 2024-05-09T08:50:31.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:30 smithi188 ceph-mon[34645]: pgmap v994: 97 pgs: 97 active+clean; 756 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 9.5 MiB/s wr, 579 op/s 2024-05-09T08:50:32.389 INFO:tasks.workunit.client.0.smithi133.stdout: 10 232249 30.22 MB/sec execute 287 sec latency 94.241 ms 2024-05-09T08:50:33.389 INFO:tasks.workunit.client.0.smithi133.stdout: 10 232796 30.21 MB/sec execute 288 sec latency 105.937 ms 2024-05-09T08:50:33.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:32 smithi160 ceph-mon[34070]: pgmap v995: 97 pgs: 97 active+clean; 758 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-05-09T08:50:33.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:32 smithi133 ceph-mon[29610]: pgmap v995: 97 pgs: 97 active+clean; 758 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-05-09T08:50:33.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:32 smithi188 ceph-mon[34645]: pgmap v995: 97 pgs: 97 active+clean; 758 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 9.9 MiB/s wr, 562 op/s 2024-05-09T08:50:34.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 233324 30.22 MB/sec execute 289 sec latency 78.074 ms 2024-05-09T08:50:35.389 INFO:tasks.workunit.client.0.smithi133.stdout: 10 233768 30.19 MB/sec execute 290 sec latency 95.869 ms 2024-05-09T08:50:35.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:34 smithi160 ceph-mon[34070]: pgmap v996: 97 pgs: 97 active+clean; 755 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 502 op/s 2024-05-09T08:50:35.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:34 smithi133 ceph-mon[29610]: pgmap v996: 97 pgs: 97 active+clean; 755 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 502 op/s 2024-05-09T08:50:35.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:34 smithi188 ceph-mon[34645]: pgmap v996: 97 pgs: 97 active+clean; 755 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.7 MiB/s wr, 502 op/s 2024-05-09T08:50:36.389 INFO:tasks.workunit.client.0.smithi133.stdout: 10 234344 30.19 MB/sec execute 291 sec latency 52.149 ms 2024-05-09T08:50:37.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 235065 30.17 MB/sec execute 292 sec latency 110.718 ms 2024-05-09T08:50:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:36 smithi160 ceph-mon[34070]: pgmap v997: 97 pgs: 97 active+clean; 766 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 621 op/s 2024-05-09T08:50:37.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:36 smithi133 ceph-mon[29610]: pgmap v997: 97 pgs: 97 active+clean; 766 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 621 op/s 2024-05-09T08:50:37.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:36 smithi188 ceph-mon[34645]: pgmap v997: 97 pgs: 97 active+clean; 766 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.0 MiB/s wr, 621 op/s 2024-05-09T08:50:38.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 235776 30.23 MB/sec execute 293 sec latency 50.602 ms 2024-05-09T08:50:39.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 236538 30.28 MB/sec execute 294 sec latency 77.825 ms 2024-05-09T08:50:39.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:38 smithi160 ceph-mon[34070]: pgmap v998: 97 pgs: 97 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-05-09T08:50:39.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:38 smithi133 ceph-mon[29610]: pgmap v998: 97 pgs: 97 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-05-09T08:50:39.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:38 smithi188 ceph-mon[34645]: pgmap v998: 97 pgs: 97 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.2 MiB/s wr, 419 op/s 2024-05-09T08:50:40.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 237066 30.29 MB/sec execute 295 sec latency 62.603 ms 2024-05-09T08:50:41.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 237614 30.28 MB/sec execute 296 sec latency 135.597 ms 2024-05-09T08:50:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:40 smithi160 ceph-mon[34070]: pgmap v999: 97 pgs: 97 active+clean; 764 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 466 op/s 2024-05-09T08:50:41.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:40 smithi133 ceph-mon[29610]: pgmap v999: 97 pgs: 97 active+clean; 764 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 466 op/s 2024-05-09T08:50:41.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:40 smithi188 ceph-mon[34645]: pgmap v999: 97 pgs: 97 active+clean; 764 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.1 MiB/s wr, 466 op/s 2024-05-09T08:50:42.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 238116 30.26 MB/sec execute 297 sec latency 62.120 ms 2024-05-09T08:50:43.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 238791 30.29 MB/sec execute 298 sec latency 46.920 ms 2024-05-09T08:50:43.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:43 smithi160 ceph-mon[34070]: pgmap v1000: 97 pgs: 97 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-05-09T08:50:43.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:43 smithi133 ceph-mon[29610]: pgmap v1000: 97 pgs: 97 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-05-09T08:50:43.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:43 smithi188 ceph-mon[34645]: pgmap v1000: 97 pgs: 97 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.9 MiB/s wr, 411 op/s 2024-05-09T08:50:44.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 239282 30.24 MB/sec execute 299 sec latency 151.392 ms 2024-05-09T08:50:45.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 239916 30.33 MB/sec execute 300 sec latency 72.090 ms 2024-05-09T08:50:45.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:45 smithi160 ceph-mon[34070]: pgmap v1001: 97 pgs: 97 active+clean; 759 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-05-09T08:50:45.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:45 smithi133 ceph-mon[29610]: pgmap v1001: 97 pgs: 97 active+clean; 759 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-05-09T08:50:45.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:45 smithi188 ceph-mon[34645]: pgmap v1001: 97 pgs: 97 active+clean; 759 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-05-09T08:50:46.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 240659 30.35 MB/sec execute 301 sec latency 69.649 ms 2024-05-09T08:50:47.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 241039 30.34 MB/sec execute 302 sec latency 195.695 ms 2024-05-09T08:50:47.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:47 smithi160 ceph-mon[34070]: pgmap v1002: 97 pgs: 97 active+clean; 767 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-05-09T08:50:47.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:47 smithi133 ceph-mon[29610]: pgmap v1002: 97 pgs: 97 active+clean; 767 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-05-09T08:50:47.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:47 smithi188 ceph-mon[34645]: pgmap v1002: 97 pgs: 97 active+clean; 767 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-05-09T08:50:48.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 241581 30.36 MB/sec execute 303 sec latency 76.586 ms 2024-05-09T08:50:48.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:48 smithi160 ceph-mon[34070]: pgmap v1003: 97 pgs: 97 active+clean; 770 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-05-09T08:50:48.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:48 smithi133 ceph-mon[29610]: pgmap v1003: 97 pgs: 97 active+clean; 770 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-05-09T08:50:48.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:48 smithi188 ceph-mon[34645]: pgmap v1003: 97 pgs: 97 active+clean; 770 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-05-09T08:50:49.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 242129 30.35 MB/sec execute 304 sec latency 113.090 ms 2024-05-09T08:50:50.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 242548 30.29 MB/sec execute 305 sec latency 141.227 ms 2024-05-09T08:50:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:50 smithi160 ceph-mon[34070]: pgmap v1004: 97 pgs: 97 active+clean; 766 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-05-09T08:50:50.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:50 smithi133 ceph-mon[29610]: pgmap v1004: 97 pgs: 97 active+clean; 766 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-05-09T08:50:50.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:50 smithi188 ceph-mon[34645]: pgmap v1004: 97 pgs: 97 active+clean; 766 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-05-09T08:50:51.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 243139 30.31 MB/sec execute 306 sec latency 111.886 ms 2024-05-09T08:50:52.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 243516 30.26 MB/sec execute 307 sec latency 428.905 ms 2024-05-09T08:50:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:52 smithi160 ceph-mon[34070]: pgmap v1005: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-05-09T08:50:52.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:52 smithi133 ceph-mon[29610]: pgmap v1005: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-05-09T08:50:52.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:52 smithi188 ceph-mon[34645]: pgmap v1005: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 474 op/s 2024-05-09T08:50:53.390 INFO:tasks.workunit.client.0.smithi133.stdout: 10 243641 30.18 MB/sec execute 308 sec latency 863.531 ms 2024-05-09T08:50:54.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 244163 30.17 MB/sec execute 309 sec latency 106.965 ms 2024-05-09T08:50:54.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:54 smithi133 ceph-mon[29610]: pgmap v1006: 97 pgs: 97 active+clean; 771 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 411 op/s 2024-05-09T08:50:54.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:54 smithi188 ceph-mon[34645]: pgmap v1006: 97 pgs: 97 active+clean; 771 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 411 op/s 2024-05-09T08:50:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:54 smithi160 ceph-mon[34070]: pgmap v1006: 97 pgs: 97 active+clean; 771 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 411 op/s 2024-05-09T08:50:55.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 244665 30.18 MB/sec execute 310 sec latency 286.835 ms 2024-05-09T08:50:56.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 245322 30.20 MB/sec execute 311 sec latency 117.138 ms 2024-05-09T08:50:56.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:56 smithi188 ceph-mon[34645]: pgmap v1007: 97 pgs: 97 active+clean; 804 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-05-09T08:50:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:56 smithi160 ceph-mon[34070]: pgmap v1007: 97 pgs: 97 active+clean; 804 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-05-09T08:50:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:56 smithi133 ceph-mon[29610]: pgmap v1007: 97 pgs: 97 active+clean; 804 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-05-09T08:50:57.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 246041 30.24 MB/sec execute 312 sec latency 92.998 ms 2024-05-09T08:50:58.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 246556 30.23 MB/sec execute 313 sec latency 67.998 ms 2024-05-09T08:50:58.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:50:58 smithi188 ceph-mon[34645]: pgmap v1008: 97 pgs: 97 active+clean; 793 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 584 op/s 2024-05-09T08:50:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:50:58 smithi160 ceph-mon[34070]: pgmap v1008: 97 pgs: 97 active+clean; 793 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 584 op/s 2024-05-09T08:50:58.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:50:58 smithi133 ceph-mon[29610]: pgmap v1008: 97 pgs: 97 active+clean; 793 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 13 MiB/s wr, 584 op/s 2024-05-09T08:50:59.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 247054 30.23 MB/sec execute 314 sec latency 121.495 ms 2024-05-09T08:51:00.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 247530 30.21 MB/sec execute 315 sec latency 82.930 ms 2024-05-09T08:51:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:00 smithi160 ceph-mon[34070]: pgmap v1009: 97 pgs: 97 active+clean; 763 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-09T08:51:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:00 smithi133 ceph-mon[29610]: pgmap v1009: 97 pgs: 97 active+clean; 763 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-09T08:51:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:00 smithi188 ceph-mon[34645]: pgmap v1009: 97 pgs: 97 active+clean; 763 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-09T08:51:01.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 248109 30.17 MB/sec execute 316 sec latency 64.324 ms 2024-05-09T08:51:02.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 248848 30.20 MB/sec execute 317 sec latency 63.153 ms 2024-05-09T08:51:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:02 smithi160 ceph-mon[34070]: pgmap v1010: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T08:51:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:02 smithi133 ceph-mon[29610]: pgmap v1010: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T08:51:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:02 smithi188 ceph-mon[34645]: pgmap v1010: 97 pgs: 97 active+clean; 767 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-05-09T08:51:03.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 249637 30.26 MB/sec execute 318 sec latency 53.375 ms 2024-05-09T08:51:04.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 250294 30.29 MB/sec execute 319 sec latency 73.440 ms 2024-05-09T08:51:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:04 smithi160 ceph-mon[34070]: pgmap v1011: 97 pgs: 97 active+clean; 768 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-09T08:51:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:04 smithi133 ceph-mon[29610]: pgmap v1011: 97 pgs: 97 active+clean; 768 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-09T08:51:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:04 smithi188 ceph-mon[34645]: pgmap v1011: 97 pgs: 97 active+clean; 768 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 567 op/s 2024-05-09T08:51:05.391 INFO:tasks.workunit.client.0.smithi133.stdout: 10 250783 30.29 MB/sec execute 320 sec latency 63.969 ms 2024-05-09T08:51:06.392 INFO:tasks.workunit.client.0.smithi133.stdout: 10 251202 30.26 MB/sec execute 321 sec latency 144.555 ms 2024-05-09T08:51:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:06 smithi160 ceph-mon[34070]: pgmap v1012: 97 pgs: 97 active+clean; 779 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-05-09T08:51:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:06 smithi133 ceph-mon[29610]: pgmap v1012: 97 pgs: 97 active+clean; 779 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-05-09T08:51:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:06 smithi188 ceph-mon[34645]: pgmap v1012: 97 pgs: 97 active+clean; 779 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 684 op/s 2024-05-09T08:51:07.392 INFO:tasks.workunit.client.0.smithi133.stdout: 10 251772 30.24 MB/sec execute 322 sec latency 74.435 ms 2024-05-09T08:51:08.392 INFO:tasks.workunit.client.0.smithi133.stdout: 10 252201 30.22 MB/sec execute 323 sec latency 248.147 ms 2024-05-09T08:51:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:08 smithi160 ceph-mon[34070]: pgmap v1013: 97 pgs: 97 active+clean; 782 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-05-09T08:51:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:08 smithi133 ceph-mon[29610]: pgmap v1013: 97 pgs: 97 active+clean; 782 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-05-09T08:51:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:08 smithi188 ceph-mon[34645]: pgmap v1013: 97 pgs: 97 active+clean; 782 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 477 op/s 2024-05-09T08:51:09.392 INFO:tasks.workunit.client.0.smithi133.stdout: 10 252794 30.20 MB/sec execute 324 sec latency 84.221 ms 2024-05-09T08:51:10.393 INFO:tasks.workunit.client.0.smithi133.stdout: 10 253450 30.22 MB/sec execute 325 sec latency 72.329 ms 2024-05-09T08:51:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:10 smithi160 ceph-mon[34070]: pgmap v1014: 97 pgs: 97 active+clean; 772 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-05-09T08:51:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:10 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:51:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:10 smithi133 ceph-mon[29610]: pgmap v1014: 97 pgs: 97 active+clean; 772 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-05-09T08:51:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:10 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:51:10.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:10 smithi188 ceph-mon[34645]: pgmap v1014: 97 pgs: 97 active+clean; 772 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.4 MiB/s wr, 459 op/s 2024-05-09T08:51:10.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:10 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:51:11.393 INFO:tasks.workunit.client.0.smithi133.stdout: 10 254301 30.28 MB/sec execute 326 sec latency 72.217 ms 2024-05-09T08:51:12.393 INFO:tasks.workunit.client.0.smithi133.stdout: 10 254782 30.31 MB/sec execute 327 sec latency 73.260 ms 2024-05-09T08:51:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:12 smithi160 ceph-mon[34070]: pgmap v1015: 97 pgs: 97 active+clean; 774 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-09T08:51:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:12 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:12 smithi133 ceph-mon[29610]: pgmap v1015: 97 pgs: 97 active+clean; 774 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-09T08:51:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:12 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:12 smithi188 ceph-mon[34645]: pgmap v1015: 97 pgs: 97 active+clean; 774 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.7 MiB/s wr, 513 op/s 2024-05-09T08:51:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:12 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:13.395 INFO:tasks.workunit.client.0.smithi133.stdout: 10 255046 30.26 MB/sec execute 328 sec latency 439.627 ms 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:13 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.171 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:13 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:13 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.395 INFO:tasks.workunit.client.0.smithi133.stdout: 10 255168 30.19 MB/sec execute 329 sec latency 898.971 ms 2024-05-09T08:51:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:14 smithi133 ceph-mon[29610]: pgmap v1016: 97 pgs: 97 active+clean; 773 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 386 op/s 2024-05-09T08:51:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:14 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:14 smithi160 ceph-mon[34070]: pgmap v1016: 97 pgs: 97 active+clean; 773 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 386 op/s 2024-05-09T08:51:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:14 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:14 smithi188 ceph-mon[34645]: pgmap v1016: 97 pgs: 97 active+clean; 773 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 386 op/s 2024-05-09T08:51:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:14 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:15.396 INFO:tasks.workunit.client.0.smithi133.stdout: 10 255501 30.14 MB/sec execute 330 sec latency 112.462 ms 2024-05-09T08:51:16.396 INFO:tasks.workunit.client.0.smithi133.stdout: 10 255824 30.09 MB/sec execute 331 sec latency 137.874 ms 2024-05-09T08:51:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:16 smithi160 ceph-mon[34070]: pgmap v1017: 97 pgs: 97 active+clean; 818 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-05-09T08:51:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:16 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:16 smithi133 ceph-mon[29610]: pgmap v1017: 97 pgs: 97 active+clean; 818 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-05-09T08:51:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:16 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:16 smithi188 ceph-mon[34645]: pgmap v1017: 97 pgs: 97 active+clean; 818 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-05-09T08:51:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:16 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:17.396 INFO:tasks.workunit.client.0.smithi133.stdout: 10 256360 30.07 MB/sec execute 332 sec latency 85.000 ms 2024-05-09T08:51:18.396 INFO:tasks.workunit.client.0.smithi133.stdout: 10 257074 30.07 MB/sec execute 333 sec latency 73.294 ms 2024-05-09T08:51:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:18 smithi160 ceph-mon[34070]: pgmap v1018: 97 pgs: 97 active+clean; 806 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-05-09T08:51:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:51:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:51:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:18 smithi133 ceph-mon[29610]: pgmap v1018: 97 pgs: 97 active+clean; 806 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-05-09T08:51:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:51:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:51:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:18 smithi188 ceph-mon[34645]: pgmap v1018: 97 pgs: 97 active+clean; 806 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-05-09T08:51:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:51:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:51:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:51:19.396 INFO:tasks.workunit.client.0.smithi133.stdout: 10 257812 30.12 MB/sec execute 334 sec latency 61.319 ms 2024-05-09T08:51:20.397 INFO:tasks.workunit.client.0.smithi133.stdout: 10 258539 30.16 MB/sec execute 335 sec latency 69.263 ms 2024-05-09T08:51:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:20 smithi160 ceph-mon[34070]: pgmap v1019: 97 pgs: 97 active+clean; 768 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 585 op/s 2024-05-09T08:51:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:20 smithi133 ceph-mon[29610]: pgmap v1019: 97 pgs: 97 active+clean; 768 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 585 op/s 2024-05-09T08:51:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:20 smithi188 ceph-mon[34645]: pgmap v1019: 97 pgs: 97 active+clean; 768 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 11 MiB/s wr, 585 op/s 2024-05-09T08:51:21.397 INFO:tasks.workunit.client.0.smithi133.stdout: 10 259146 30.19 MB/sec execute 336 sec latency 65.298 ms 2024-05-09T08:51:22.397 INFO:tasks.workunit.client.0.smithi133.stdout: 10 259613 30.18 MB/sec execute 337 sec latency 142.866 ms 2024-05-09T08:51:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:22 smithi160 ceph-mon[34070]: pgmap v1020: 97 pgs: 97 active+clean; 752 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 653 op/s 2024-05-09T08:51:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:22 smithi133 ceph-mon[29610]: pgmap v1020: 97 pgs: 97 active+clean; 752 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 653 op/s 2024-05-09T08:51:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:22 smithi188 ceph-mon[34645]: pgmap v1020: 97 pgs: 97 active+clean; 752 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 11 MiB/s wr, 653 op/s 2024-05-09T08:51:23.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 259962 30.14 MB/sec execute 338 sec latency 165.095 ms 2024-05-09T08:51:24.397 INFO:tasks.workunit.client.0.smithi133.stdout: 10 260439 30.11 MB/sec execute 339 sec latency 111.336 ms 2024-05-09T08:51:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:24 smithi188 ceph-mon[34645]: pgmap v1021: 97 pgs: 97 active+clean; 753 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.2 MiB/s wr, 536 op/s 2024-05-09T08:51:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:24 smithi160 ceph-mon[34070]: pgmap v1021: 97 pgs: 97 active+clean; 753 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.2 MiB/s wr, 536 op/s 2024-05-09T08:51:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:24 smithi133 ceph-mon[29610]: pgmap v1021: 97 pgs: 97 active+clean; 753 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.2 MiB/s wr, 536 op/s 2024-05-09T08:51:25.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 261027 30.09 MB/sec execute 340 sec latency 78.806 ms 2024-05-09T08:51:26.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 261777 30.12 MB/sec execute 341 sec latency 55.307 ms 2024-05-09T08:51:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:26 smithi160 ceph-mon[34070]: pgmap v1022: 97 pgs: 97 active+clean; 762 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.7 MiB/s wr, 634 op/s 2024-05-09T08:51:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:26 smithi133 ceph-mon[29610]: pgmap v1022: 97 pgs: 97 active+clean; 762 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.7 MiB/s wr, 634 op/s 2024-05-09T08:51:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:26 smithi188 ceph-mon[34645]: pgmap v1022: 97 pgs: 97 active+clean; 762 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 9.7 MiB/s wr, 634 op/s 2024-05-09T08:51:27.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 262641 30.15 MB/sec execute 342 sec latency 58.421 ms 2024-05-09T08:51:28.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 263326 30.24 MB/sec execute 343 sec latency 56.407 ms 2024-05-09T08:51:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:28 smithi160 ceph-mon[34070]: pgmap v1023: 97 pgs: 97 active+clean; 761 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-05-09T08:51:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:28 smithi133 ceph-mon[29610]: pgmap v1023: 97 pgs: 97 active+clean; 761 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-05-09T08:51:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:28 smithi188 ceph-mon[34645]: pgmap v1023: 97 pgs: 97 active+clean; 761 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 489 op/s 2024-05-09T08:51:29.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 263634 30.21 MB/sec execute 344 sec latency 203.364 ms 2024-05-09T08:51:30.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 263931 30.15 MB/sec execute 345 sec latency 155.529 ms 2024-05-09T08:51:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:30 smithi160 ceph-mon[34070]: pgmap v1024: 97 pgs: 97 active+clean; 775 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-09T08:51:31.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:30 smithi133 ceph-mon[29610]: pgmap v1024: 97 pgs: 97 active+clean; 775 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-09T08:51:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:30 smithi188 ceph-mon[34645]: pgmap v1024: 97 pgs: 97 active+clean; 775 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 504 op/s 2024-05-09T08:51:31.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 264356 30.10 MB/sec execute 346 sec latency 134.668 ms 2024-05-09T08:51:32.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 264829 30.10 MB/sec execute 347 sec latency 90.868 ms 2024-05-09T08:51:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:32 smithi160 ceph-mon[34070]: pgmap v1025: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 478 op/s 2024-05-09T08:51:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:32 smithi133 ceph-mon[29610]: pgmap v1025: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 478 op/s 2024-05-09T08:51:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:32 smithi188 ceph-mon[34645]: pgmap v1025: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.5 MiB/s wr, 478 op/s 2024-05-09T08:51:33.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 265399 30.08 MB/sec execute 348 sec latency 74.757 ms 2024-05-09T08:51:34.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 265962 30.05 MB/sec execute 349 sec latency 96.760 ms 2024-05-09T08:51:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:34 smithi160 ceph-mon[34070]: pgmap v1026: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 370 op/s 2024-05-09T08:51:35.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:34 smithi133 ceph-mon[29610]: pgmap v1026: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 370 op/s 2024-05-09T08:51:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:34 smithi188 ceph-mon[34645]: pgmap v1026: 97 pgs: 97 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 370 op/s 2024-05-09T08:51:35.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 266068 29.99 MB/sec execute 350 sec latency 646.224 ms 2024-05-09T08:51:36.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 266611 29.99 MB/sec execute 351 sec latency 378.400 ms 2024-05-09T08:51:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:36 smithi160 ceph-mon[34070]: pgmap v1027: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-05-09T08:51:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:36 smithi133 ceph-mon[29610]: pgmap v1027: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-05-09T08:51:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:36 smithi188 ceph-mon[34645]: pgmap v1027: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.5 MiB/s wr, 519 op/s 2024-05-09T08:51:37.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 267191 30.03 MB/sec execute 352 sec latency 81.208 ms 2024-05-09T08:51:38.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 267799 30.07 MB/sec execute 353 sec latency 53.933 ms 2024-05-09T08:51:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:38 smithi160 ceph-mon[34070]: pgmap v1028: 97 pgs: 97 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 470 op/s 2024-05-09T08:51:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:38 smithi133 ceph-mon[29610]: pgmap v1028: 97 pgs: 97 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 470 op/s 2024-05-09T08:51:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:38 smithi188 ceph-mon[34645]: pgmap v1028: 97 pgs: 97 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.5 MiB/s wr, 470 op/s 2024-05-09T08:51:39.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 268190 30.03 MB/sec execute 354 sec latency 53.440 ms 2024-05-09T08:51:40.398 INFO:tasks.workunit.client.0.smithi133.stdout: 10 268569 29.99 MB/sec execute 355 sec latency 140.084 ms 2024-05-09T08:51:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:40 smithi160 ceph-mon[34070]: pgmap v1029: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 572 op/s 2024-05-09T08:51:41.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:40 smithi133 ceph-mon[29610]: pgmap v1029: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 572 op/s 2024-05-09T08:51:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:40 smithi188 ceph-mon[34645]: pgmap v1029: 97 pgs: 97 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 572 op/s 2024-05-09T08:51:41.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 269084 29.97 MB/sec execute 356 sec latency 90.421 ms 2024-05-09T08:51:42.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 269633 29.95 MB/sec execute 357 sec latency 177.408 ms 2024-05-09T08:51:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:42 smithi160 ceph-mon[34070]: pgmap v1030: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-09T08:51:43.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:42 smithi133 ceph-mon[29610]: pgmap v1030: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-09T08:51:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:42 smithi188 ceph-mon[34645]: pgmap v1030: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 619 op/s 2024-05-09T08:51:43.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 270217 29.96 MB/sec execute 358 sec latency 77.382 ms 2024-05-09T08:51:44.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 270926 30.00 MB/sec execute 359 sec latency 56.375 ms 2024-05-09T08:51:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:44 smithi160 ceph-mon[34070]: pgmap v1031: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T08:51:45.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:44 smithi133 ceph-mon[29610]: pgmap v1031: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T08:51:45.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:44 smithi188 ceph-mon[34645]: pgmap v1031: 97 pgs: 97 active+clean; 782 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.4 MiB/s wr, 534 op/s 2024-05-09T08:51:45.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 271579 30.03 MB/sec execute 360 sec latency 59.372 ms 2024-05-09T08:51:46.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 272100 30.02 MB/sec execute 361 sec latency 89.302 ms 2024-05-09T08:51:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:46 smithi160 ceph-mon[34070]: pgmap v1032: 97 pgs: 97 active+clean; 773 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-05-09T08:51:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:46 smithi133 ceph-mon[29610]: pgmap v1032: 97 pgs: 97 active+clean; 773 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-05-09T08:51:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:46 smithi188 ceph-mon[34645]: pgmap v1032: 97 pgs: 97 active+clean; 773 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 10 MiB/s wr, 661 op/s 2024-05-09T08:51:47.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 272472 30.00 MB/sec execute 362 sec latency 272.550 ms 2024-05-09T08:51:48.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 272867 29.97 MB/sec execute 363 sec latency 141.331 ms 2024-05-09T08:51:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:48 smithi160 ceph-mon[34070]: pgmap v1033: 97 pgs: 97 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.7 MiB/s wr, 524 op/s 2024-05-09T08:51:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:48 smithi133 ceph-mon[29610]: pgmap v1033: 97 pgs: 97 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.7 MiB/s wr, 524 op/s 2024-05-09T08:51:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:48 smithi188 ceph-mon[34645]: pgmap v1033: 97 pgs: 97 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.7 MiB/s wr, 524 op/s 2024-05-09T08:51:49.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 273419 29.96 MB/sec execute 364 sec latency 91.566 ms 2024-05-09T08:51:50.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 274160 29.96 MB/sec execute 365 sec latency 52.205 ms 2024-05-09T08:51:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:50 smithi160 ceph-mon[34070]: pgmap v1034: 97 pgs: 97 active+clean; 774 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2024-05-09T08:51:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:50 smithi133 ceph-mon[29610]: pgmap v1034: 97 pgs: 97 active+clean; 774 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2024-05-09T08:51:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:50 smithi188 ceph-mon[34645]: pgmap v1034: 97 pgs: 97 active+clean; 774 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.2 MiB/s wr, 501 op/s 2024-05-09T08:51:51.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 275044 30.04 MB/sec execute 366 sec latency 52.056 ms 2024-05-09T08:51:52.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 275698 30.07 MB/sec execute 367 sec latency 42.779 ms 2024-05-09T08:51:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:52 smithi160 ceph-mon[34070]: pgmap v1035: 97 pgs: 97 active+clean; 784 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-09T08:51:53.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:52 smithi133 ceph-mon[29610]: pgmap v1035: 97 pgs: 97 active+clean; 784 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-09T08:51:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:52 smithi188 ceph-mon[34645]: pgmap v1035: 97 pgs: 97 active+clean; 784 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.7 MiB/s wr, 472 op/s 2024-05-09T08:51:53.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 276147 30.04 MB/sec execute 368 sec latency 77.613 ms 2024-05-09T08:51:54.399 INFO:tasks.workunit.client.0.smithi133.stdout: 10 276646 30.03 MB/sec execute 369 sec latency 80.525 ms 2024-05-09T08:51:55.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:54 smithi160 ceph-mon[34070]: pgmap v1036: 97 pgs: 97 active+clean; 785 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-05-09T08:51:55.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:54 smithi133 ceph-mon[29610]: pgmap v1036: 97 pgs: 97 active+clean; 785 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-05-09T08:51:55.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:54 smithi188 ceph-mon[34645]: pgmap v1036: 97 pgs: 97 active+clean; 785 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-05-09T08:51:55.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 277178 30.03 MB/sec execute 370 sec latency 158.011 ms 2024-05-09T08:51:56.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 277434 29.97 MB/sec execute 371 sec latency 491.734 ms 2024-05-09T08:51:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:56 smithi160 ceph-mon[34070]: pgmap v1037: 97 pgs: 97 active+clean; 794 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-05-09T08:51:57.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:56 smithi133 ceph-mon[29610]: pgmap v1037: 97 pgs: 97 active+clean; 794 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-05-09T08:51:57.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:56 smithi188 ceph-mon[34645]: pgmap v1037: 97 pgs: 97 active+clean; 794 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.8 MiB/s wr, 445 op/s 2024-05-09T08:51:57.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 278009 29.95 MB/sec execute 372 sec latency 111.478 ms 2024-05-09T08:51:58.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 278751 30.01 MB/sec execute 373 sec latency 49.734 ms 2024-05-09T08:51:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:51:58 smithi160 ceph-mon[34070]: pgmap v1038: 97 pgs: 97 active+clean; 807 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-05-09T08:51:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:51:58 smithi133 ceph-mon[29610]: pgmap v1038: 97 pgs: 97 active+clean; 807 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-05-09T08:51:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:51:58 smithi188 ceph-mon[34645]: pgmap v1038: 97 pgs: 97 active+clean; 807 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-05-09T08:51:59.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 279292 30.02 MB/sec execute 374 sec latency 55.219 ms 2024-05-09T08:52:00.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 279799 30.01 MB/sec execute 375 sec latency 89.870 ms 2024-05-09T08:52:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:00 smithi160 ceph-mon[34070]: pgmap v1039: 97 pgs: 97 active+clean; 811 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 494 op/s 2024-05-09T08:52:01.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:00 smithi133 ceph-mon[29610]: pgmap v1039: 97 pgs: 97 active+clean; 811 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 494 op/s 2024-05-09T08:52:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:00 smithi188 ceph-mon[34645]: pgmap v1039: 97 pgs: 97 active+clean; 811 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 494 op/s 2024-05-09T08:52:01.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 280201 29.97 MB/sec execute 376 sec latency 309.395 ms 2024-05-09T08:52:02.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 280351 29.90 MB/sec execute 377 sec latency 852.257 ms 2024-05-09T08:52:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:02 smithi160 ceph-mon[34070]: pgmap v1040: 97 pgs: 97 active+clean; 796 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2024-05-09T08:52:03.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:02 smithi133 ceph-mon[29610]: pgmap v1040: 97 pgs: 97 active+clean; 796 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2024-05-09T08:52:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:02 smithi188 ceph-mon[34645]: pgmap v1040: 97 pgs: 97 active+clean; 796 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 7.6 MiB/s wr, 529 op/s 2024-05-09T08:52:03.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 280774 29.90 MB/sec execute 378 sec latency 143.610 ms 2024-05-09T08:52:04.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 281295 29.90 MB/sec execute 379 sec latency 77.981 ms 2024-05-09T08:52:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:04 smithi160 ceph-mon[34070]: pgmap v1041: 97 pgs: 97 active+clean; 795 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.0 MiB/s wr, 453 op/s 2024-05-09T08:52:05.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:04 smithi133 ceph-mon[29610]: pgmap v1041: 97 pgs: 97 active+clean; 795 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.0 MiB/s wr, 453 op/s 2024-05-09T08:52:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:04 smithi188 ceph-mon[34645]: pgmap v1041: 97 pgs: 97 active+clean; 795 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.0 MiB/s wr, 453 op/s 2024-05-09T08:52:05.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 282002 29.90 MB/sec execute 380 sec latency 65.197 ms 2024-05-09T08:52:06.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 282743 29.94 MB/sec execute 381 sec latency 67.336 ms 2024-05-09T08:52:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:06 smithi160 ceph-mon[34070]: pgmap v1042: 97 pgs: 97 active+clean; 780 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-05-09T08:52:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:06 smithi133 ceph-mon[29610]: pgmap v1042: 97 pgs: 97 active+clean; 780 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-05-09T08:52:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:06 smithi188 ceph-mon[34645]: pgmap v1042: 97 pgs: 97 active+clean; 780 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 678 op/s 2024-05-09T08:52:07.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 283290 29.96 MB/sec execute 382 sec latency 96.106 ms 2024-05-09T08:52:08.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 283847 29.95 MB/sec execute 383 sec latency 97.658 ms 2024-05-09T08:52:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:08 smithi160 ceph-mon[34070]: pgmap v1043: 97 pgs: 97 active+clean; 758 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-05-09T08:52:09.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:08 smithi133 ceph-mon[29610]: pgmap v1043: 97 pgs: 97 active+clean; 758 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-05-09T08:52:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:08 smithi188 ceph-mon[34645]: pgmap v1043: 97 pgs: 97 active+clean; 758 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 628 op/s 2024-05-09T08:52:09.400 INFO:tasks.workunit.client.0.smithi133.stdout: 10 284216 29.93 MB/sec execute 384 sec latency 99.854 ms 2024-05-09T08:52:10.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 284583 29.89 MB/sec execute 385 sec latency 87.230 ms 2024-05-09T08:52:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:10 smithi160 ceph-mon[34070]: pgmap v1044: 97 pgs: 97 active+clean; 762 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:52:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:10 smithi133 ceph-mon[29610]: pgmap v1044: 97 pgs: 97 active+clean; 762 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:52:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:10 smithi188 ceph-mon[34645]: pgmap v1044: 97 pgs: 97 active+clean; 762 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 695 op/s 2024-05-09T08:52:11.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 285224 29.88 MB/sec execute 386 sec latency 129.077 ms 2024-05-09T08:52:12.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 285949 29.92 MB/sec execute 387 sec latency 41.043 ms 2024-05-09T08:52:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:12 smithi160 ceph-mon[34070]: pgmap v1045: 97 pgs: 97 active+clean; 763 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-05-09T08:52:13.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:12 smithi133 ceph-mon[29610]: pgmap v1045: 97 pgs: 97 active+clean; 763 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-05-09T08:52:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:12 smithi188 ceph-mon[34645]: pgmap v1045: 97 pgs: 97 active+clean; 763 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 9.5 MiB/s wr, 569 op/s 2024-05-09T08:52:13.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 286716 29.94 MB/sec execute 388 sec latency 56.328 ms 2024-05-09T08:52:14.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 287330 29.98 MB/sec execute 389 sec latency 58.791 ms 2024-05-09T08:52:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:14 smithi160 ceph-mon[34070]: pgmap v1046: 97 pgs: 97 active+clean; 764 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.9 MiB/s wr, 506 op/s 2024-05-09T08:52:15.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:14 smithi133 ceph-mon[29610]: pgmap v1046: 97 pgs: 97 active+clean; 764 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.9 MiB/s wr, 506 op/s 2024-05-09T08:52:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:14 smithi188 ceph-mon[34645]: pgmap v1046: 97 pgs: 97 active+clean; 764 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 7.9 MiB/s wr, 506 op/s 2024-05-09T08:52:15.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 287846 29.99 MB/sec execute 390 sec latency 64.520 ms 2024-05-09T08:52:16.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 288192 29.93 MB/sec execute 391 sec latency 101.479 ms 2024-05-09T08:52:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:16 smithi160 ceph-mon[34070]: pgmap v1047: 97 pgs: 97 active+clean; 776 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2024-05-09T08:52:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:16 smithi133 ceph-mon[29610]: pgmap v1047: 97 pgs: 97 active+clean; 776 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2024-05-09T08:52:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:16 smithi188 ceph-mon[34645]: pgmap v1047: 97 pgs: 97 active+clean; 776 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 604 op/s 2024-05-09T08:52:17.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 288490 29.90 MB/sec execute 392 sec latency 244.004 ms 2024-05-09T08:52:18.401 INFO:tasks.workunit.client.0.smithi133.stdout: 10 288907 29.85 MB/sec execute 393 sec latency 196.437 ms 2024-05-09T08:52:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:18 smithi160 ceph-mon[34070]: pgmap v1048: 97 pgs: 97 active+clean; 777 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-05-09T08:52:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:52:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:18 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:18 smithi133 ceph-mon[29610]: pgmap v1048: 97 pgs: 97 active+clean; 777 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-05-09T08:52:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:52:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:18 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:18 smithi188 ceph-mon[34645]: pgmap v1048: 97 pgs: 97 active+clean; 777 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 384 op/s 2024-05-09T08:52:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:52:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:18 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:19.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 289555 29.86 MB/sec execute 394 sec latency 63.338 ms 2024-05-09T08:52:20.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:52:20.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:52:20.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:19 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:52:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:52:20.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:19 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:52:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:52:20.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:19 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:52:20.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 290221 29.90 MB/sec execute 395 sec latency 60.976 ms 2024-05-09T08:52:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:20 smithi160 ceph-mon[34070]: pgmap v1049: 97 pgs: 97 active+clean; 784 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-05-09T08:52:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:20 smithi133 ceph-mon[29610]: pgmap v1049: 97 pgs: 97 active+clean; 784 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-05-09T08:52:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:20 smithi188 ceph-mon[34645]: pgmap v1049: 97 pgs: 97 active+clean; 784 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-05-09T08:52:21.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 290836 29.91 MB/sec execute 396 sec latency 49.818 ms 2024-05-09T08:52:22.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 291234 29.90 MB/sec execute 397 sec latency 333.057 ms 2024-05-09T08:52:23.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:22 smithi160 ceph-mon[34070]: pgmap v1050: 97 pgs: 97 active+clean; 789 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-05-09T08:52:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:22 smithi133 ceph-mon[29610]: pgmap v1050: 97 pgs: 97 active+clean; 789 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-05-09T08:52:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:22 smithi188 ceph-mon[34645]: pgmap v1050: 97 pgs: 97 active+clean; 789 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 411 op/s 2024-05-09T08:52:23.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 291377 29.84 MB/sec execute 398 sec latency 863.228 ms 2024-05-09T08:52:24.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 291793 29.82 MB/sec execute 399 sec latency 153.609 ms 2024-05-09T08:52:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:24 smithi133 ceph-mon[29610]: pgmap v1051: 97 pgs: 97 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-09T08:52:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:24 smithi188 ceph-mon[34645]: pgmap v1051: 97 pgs: 97 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-09T08:52:25.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 292237 29.82 MB/sec execute 400 sec latency 87.850 ms 2024-05-09T08:52:25.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:24 smithi160 ceph-mon[34070]: pgmap v1051: 97 pgs: 97 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.4 MiB/s wr, 430 op/s 2024-05-09T08:52:26.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 292756 29.80 MB/sec execute 401 sec latency 105.763 ms 2024-05-09T08:52:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:26 smithi133 ceph-mon[29610]: pgmap v1052: 97 pgs: 97 active+clean; 804 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T08:52:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:26 smithi188 ceph-mon[34645]: pgmap v1052: 97 pgs: 97 active+clean; 804 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T08:52:27.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 293332 29.78 MB/sec execute 402 sec latency 96.045 ms 2024-05-09T08:52:27.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:26 smithi160 ceph-mon[34070]: pgmap v1052: 97 pgs: 97 active+clean; 804 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 10 MiB/s wr, 614 op/s 2024-05-09T08:52:28.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 294064 29.80 MB/sec execute 403 sec latency 70.304 ms 2024-05-09T08:52:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:28 smithi133 ceph-mon[29610]: pgmap v1053: 97 pgs: 97 active+clean; 800 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 578 op/s 2024-05-09T08:52:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:28 smithi188 ceph-mon[34645]: pgmap v1053: 97 pgs: 97 active+clean; 800 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 578 op/s 2024-05-09T08:52:29.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 294841 29.84 MB/sec execute 404 sec latency 63.607 ms 2024-05-09T08:52:29.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:28 smithi160 ceph-mon[34070]: pgmap v1053: 97 pgs: 97 active+clean; 800 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 11 MiB/s wr, 578 op/s 2024-05-09T08:52:30.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 295458 29.88 MB/sec execute 405 sec latency 51.311 ms 2024-05-09T08:52:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:30 smithi188 ceph-mon[34645]: pgmap v1054: 97 pgs: 97 active+clean; 778 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-09T08:52:31.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 295947 29.86 MB/sec execute 406 sec latency 64.435 ms 2024-05-09T08:52:31.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:30 smithi160 ceph-mon[34070]: pgmap v1054: 97 pgs: 97 active+clean; 778 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-09T08:52:31.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:30 smithi133 ceph-mon[29610]: pgmap v1054: 97 pgs: 97 active+clean; 778 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 11 MiB/s wr, 616 op/s 2024-05-09T08:52:32.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 296541 29.86 MB/sec execute 407 sec latency 112.540 ms 2024-05-09T08:52:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:32 smithi160 ceph-mon[34070]: pgmap v1055: 97 pgs: 97 active+clean; 770 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-05-09T08:52:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:32 smithi188 ceph-mon[34645]: pgmap v1055: 97 pgs: 97 active+clean; 770 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-05-09T08:52:33.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 297030 29.86 MB/sec execute 408 sec latency 81.737 ms 2024-05-09T08:52:33.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:32 smithi133 ceph-mon[29610]: pgmap v1055: 97 pgs: 97 active+clean; 770 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 565 op/s 2024-05-09T08:52:34.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 297605 29.84 MB/sec execute 409 sec latency 73.268 ms 2024-05-09T08:52:35.402 INFO:tasks.workunit.client.0.smithi133.stdout: 10 298166 29.84 MB/sec execute 410 sec latency 138.683 ms 2024-05-09T08:52:35.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:34 smithi160 ceph-mon[34070]: pgmap v1056: 97 pgs: 97 active+clean; 775 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.5 MiB/s wr, 514 op/s 2024-05-09T08:52:35.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:34 smithi133 ceph-mon[29610]: pgmap v1056: 97 pgs: 97 active+clean; 775 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.5 MiB/s wr, 514 op/s 2024-05-09T08:52:35.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:34 smithi188 ceph-mon[34645]: pgmap v1056: 97 pgs: 97 active+clean; 775 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.5 MiB/s wr, 514 op/s 2024-05-09T08:52:36.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 298902 29.87 MB/sec execute 411 sec latency 55.549 ms 2024-05-09T08:52:37.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 299547 29.89 MB/sec execute 412 sec latency 51.205 ms 2024-05-09T08:52:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:36 smithi160 ceph-mon[34070]: pgmap v1057: 97 pgs: 97 active+clean; 784 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.4 MiB/s wr, 583 op/s 2024-05-09T08:52:37.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:36 smithi133 ceph-mon[29610]: pgmap v1057: 97 pgs: 97 active+clean; 784 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.4 MiB/s wr, 583 op/s 2024-05-09T08:52:37.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:36 smithi188 ceph-mon[34645]: pgmap v1057: 97 pgs: 97 active+clean; 784 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.4 MiB/s wr, 583 op/s 2024-05-09T08:52:38.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 300016 29.89 MB/sec execute 413 sec latency 88.568 ms 2024-05-09T08:52:39.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 300492 29.88 MB/sec execute 414 sec latency 115.214 ms 2024-05-09T08:52:39.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:38 smithi160 ceph-mon[34070]: pgmap v1058: 97 pgs: 97 active+clean; 791 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 462 op/s 2024-05-09T08:52:39.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:38 smithi133 ceph-mon[29610]: pgmap v1058: 97 pgs: 97 active+clean; 791 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 462 op/s 2024-05-09T08:52:39.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:38 smithi188 ceph-mon[34645]: pgmap v1058: 97 pgs: 97 active+clean; 791 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.9 MiB/s wr, 462 op/s 2024-05-09T08:52:40.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 300963 29.87 MB/sec execute 415 sec latency 114.556 ms 2024-05-09T08:52:41.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 301441 29.84 MB/sec execute 416 sec latency 71.520 ms 2024-05-09T08:52:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:40 smithi160 ceph-mon[34070]: pgmap v1059: 97 pgs: 97 active+clean; 792 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:52:41.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:40 smithi133 ceph-mon[29610]: pgmap v1059: 97 pgs: 97 active+clean; 792 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:52:41.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:40 smithi188 ceph-mon[34645]: pgmap v1059: 97 pgs: 97 active+clean; 792 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:52:42.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 301965 29.83 MB/sec execute 417 sec latency 87.632 ms 2024-05-09T08:52:43.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 302348 29.81 MB/sec execute 418 sec latency 299.752 ms 2024-05-09T08:52:43.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:42 smithi160 ceph-mon[34070]: pgmap v1060: 97 pgs: 97 active+clean; 799 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-05-09T08:52:43.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:42 smithi133 ceph-mon[29610]: pgmap v1060: 97 pgs: 97 active+clean; 799 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-05-09T08:52:43.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:42 smithi188 ceph-mon[34645]: pgmap v1060: 97 pgs: 97 active+clean; 799 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.5 MiB/s wr, 411 op/s 2024-05-09T08:52:44.403 INFO:tasks.workunit.client.0.smithi133.stdout: 10 302665 29.78 MB/sec execute 419 sec latency 253.420 ms 2024-05-09T08:52:45.234 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:44 smithi188 ceph-mon[34645]: pgmap v1061: 97 pgs: 97 active+clean; 801 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-09T08:52:45.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 303201 29.80 MB/sec execute 420 sec latency 105.887 ms 2024-05-09T08:52:45.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:44 smithi160 ceph-mon[34070]: pgmap v1061: 97 pgs: 97 active+clean; 801 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-09T08:52:45.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:44 smithi133 ceph-mon[29610]: pgmap v1061: 97 pgs: 97 active+clean; 801 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-05-09T08:52:46.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 303887 29.79 MB/sec execute 421 sec latency 37.476 ms 2024-05-09T08:52:47.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 304374 29.80 MB/sec execute 422 sec latency 243.581 ms 2024-05-09T08:52:47.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:46 smithi160 ceph-mon[34070]: pgmap v1062: 97 pgs: 97 active+clean; 810 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-05-09T08:52:47.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:46 smithi133 ceph-mon[29610]: pgmap v1062: 97 pgs: 97 active+clean; 810 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-05-09T08:52:47.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:46 smithi188 ceph-mon[34645]: pgmap v1062: 97 pgs: 97 active+clean; 810 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.2 MiB/s wr, 476 op/s 2024-05-09T08:52:48.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 304808 29.80 MB/sec execute 423 sec latency 187.324 ms 2024-05-09T08:52:49.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 304823 29.73 MB/sec execute 424 sec latency 588.252 ms 2024-05-09T08:52:49.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:48 smithi160 ceph-mon[34070]: pgmap v1063: 97 pgs: 97 active+clean; 811 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-05-09T08:52:49.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:48 smithi133 ceph-mon[29610]: pgmap v1063: 97 pgs: 97 active+clean; 811 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-05-09T08:52:49.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:48 smithi188 ceph-mon[34645]: pgmap v1063: 97 pgs: 97 active+clean; 811 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 419 op/s 2024-05-09T08:52:50.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 305047 29.68 MB/sec execute 425 sec latency 573.516 ms 2024-05-09T08:52:51.229 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:50 smithi133 ceph-mon[29610]: pgmap v1064: 97 pgs: 97 active+clean; 823 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2024-05-09T08:52:51.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 305468 29.66 MB/sec execute 426 sec latency 101.541 ms 2024-05-09T08:52:51.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:50 smithi160 ceph-mon[34070]: pgmap v1064: 97 pgs: 97 active+clean; 823 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2024-05-09T08:52:51.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:50 smithi188 ceph-mon[34645]: pgmap v1064: 97 pgs: 97 active+clean; 823 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2024-05-09T08:52:52.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 306029 29.65 MB/sec execute 427 sec latency 285.932 ms 2024-05-09T08:52:53.404 INFO:tasks.workunit.client.0.smithi133.stdout: 10 306702 29.67 MB/sec execute 428 sec latency 75.107 ms 2024-05-09T08:52:53.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:52 smithi160 ceph-mon[34070]: pgmap v1065: 97 pgs: 97 active+clean; 843 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-09T08:52:53.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:52 smithi133 ceph-mon[29610]: pgmap v1065: 97 pgs: 97 active+clean; 843 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-09T08:52:53.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:52 smithi188 ceph-mon[34645]: pgmap v1065: 97 pgs: 97 active+clean; 843 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-05-09T08:52:54.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 307233 29.67 MB/sec execute 429 sec latency 93.107 ms 2024-05-09T08:52:55.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 307815 29.68 MB/sec execute 430 sec latency 51.031 ms 2024-05-09T08:52:55.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:54 smithi160 ceph-mon[34070]: pgmap v1066: 97 pgs: 97 active+clean; 840 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 543 op/s 2024-05-09T08:52:55.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:54 smithi133 ceph-mon[29610]: pgmap v1066: 97 pgs: 97 active+clean; 840 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 543 op/s 2024-05-09T08:52:55.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:54 smithi188 ceph-mon[34645]: pgmap v1066: 97 pgs: 97 active+clean; 840 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 543 op/s 2024-05-09T08:52:56.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 308405 29.68 MB/sec execute 431 sec latency 102.098 ms 2024-05-09T08:52:57.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 309013 29.69 MB/sec execute 432 sec latency 50.614 ms 2024-05-09T08:52:57.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:56 smithi160 ceph-mon[34070]: pgmap v1067: 97 pgs: 97 active+clean; 785 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 708 op/s 2024-05-09T08:52:57.428 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:56 smithi133 ceph-mon[29610]: pgmap v1067: 97 pgs: 97 active+clean; 785 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 708 op/s 2024-05-09T08:52:57.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:56 smithi188 ceph-mon[34645]: pgmap v1067: 97 pgs: 97 active+clean; 785 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 13 MiB/s wr, 708 op/s 2024-05-09T08:52:58.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 309568 29.69 MB/sec execute 433 sec latency 117.426 ms 2024-05-09T08:52:59.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 310099 29.69 MB/sec execute 434 sec latency 55.731 ms 2024-05-09T08:52:59.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:52:58 smithi160 ceph-mon[34070]: pgmap v1068: 97 pgs: 97 active+clean; 779 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-05-09T08:52:59.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:52:58 smithi133 ceph-mon[29610]: pgmap v1068: 97 pgs: 97 active+clean; 779 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-05-09T08:52:59.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:52:58 smithi188 ceph-mon[34645]: pgmap v1068: 97 pgs: 97 active+clean; 779 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2024-05-09T08:53:00.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 310673 29.69 MB/sec execute 435 sec latency 103.857 ms 2024-05-09T08:53:01.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 311304 29.69 MB/sec execute 436 sec latency 51.322 ms 2024-05-09T08:53:01.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:00 smithi160 ceph-mon[34070]: pgmap v1069: 97 pgs: 97 active+clean; 782 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 587 op/s 2024-05-09T08:53:01.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:00 smithi133 ceph-mon[29610]: pgmap v1069: 97 pgs: 97 active+clean; 782 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 587 op/s 2024-05-09T08:53:01.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:00 smithi188 ceph-mon[34645]: pgmap v1069: 97 pgs: 97 active+clean; 782 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 587 op/s 2024-05-09T08:53:02.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 311930 29.70 MB/sec execute 437 sec latency 53.843 ms 2024-05-09T08:53:03.405 INFO:tasks.workunit.client.0.smithi133.stdout: 10 312567 29.73 MB/sec execute 438 sec latency 60.074 ms 2024-05-09T08:53:03.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:02 smithi160 ceph-mon[34070]: pgmap v1070: 97 pgs: 97 active+clean; 775 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-05-09T08:53:03.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:02 smithi133 ceph-mon[29610]: pgmap v1070: 97 pgs: 97 active+clean; 775 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-05-09T08:53:03.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:02 smithi188 ceph-mon[34645]: pgmap v1070: 97 pgs: 97 active+clean; 775 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-05-09T08:53:04.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 313122 29.72 MB/sec execute 439 sec latency 74.896 ms 2024-05-09T08:53:05.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 313675 29.72 MB/sec execute 440 sec latency 69.186 ms 2024-05-09T08:53:05.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:04 smithi160 ceph-mon[34070]: pgmap v1071: 97 pgs: 97 active+clean; 780 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-05-09T08:53:05.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:04 smithi133 ceph-mon[29610]: pgmap v1071: 97 pgs: 97 active+clean; 780 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-05-09T08:53:05.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:04 smithi188 ceph-mon[34645]: pgmap v1071: 97 pgs: 97 active+clean; 780 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 493 op/s 2024-05-09T08:53:06.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 314334 29.73 MB/sec execute 441 sec latency 55.797 ms 2024-05-09T08:53:07.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 314964 29.75 MB/sec execute 442 sec latency 64.640 ms 2024-05-09T08:53:07.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:07 smithi160 ceph-mon[34070]: pgmap v1072: 97 pgs: 97 active+clean; 786 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-05-09T08:53:07.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:07 smithi133 ceph-mon[29610]: pgmap v1072: 97 pgs: 97 active+clean; 786 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-05-09T08:53:07.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:07 smithi188 ceph-mon[34645]: pgmap v1072: 97 pgs: 97 active+clean; 786 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-05-09T08:53:08.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 315451 29.72 MB/sec execute 443 sec latency 134.459 ms 2024-05-09T08:53:09.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 315981 29.72 MB/sec execute 444 sec latency 210.166 ms 2024-05-09T08:53:09.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:09 smithi160 ceph-mon[34070]: pgmap v1073: 97 pgs: 97 active+clean; 793 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-05-09T08:53:09.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:09 smithi133 ceph-mon[29610]: pgmap v1073: 97 pgs: 97 active+clean; 793 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-05-09T08:53:09.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:09 smithi188 ceph-mon[34645]: pgmap v1073: 97 pgs: 97 active+clean; 793 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-05-09T08:53:10.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 316392 29.71 MB/sec execute 445 sec latency 365.250 ms 2024-05-09T08:53:11.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 317013 29.72 MB/sec execute 446 sec latency 62.590 ms 2024-05-09T08:53:11.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:11 smithi160 ceph-mon[34070]: pgmap v1074: 97 pgs: 97 active+clean; 797 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 342 op/s 2024-05-09T08:53:11.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:11 smithi133 ceph-mon[29610]: pgmap v1074: 97 pgs: 97 active+clean; 797 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 342 op/s 2024-05-09T08:53:11.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:11 smithi188 ceph-mon[34645]: pgmap v1074: 97 pgs: 97 active+clean; 797 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 342 op/s 2024-05-09T08:53:12.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 317427 29.71 MB/sec execute 447 sec latency 314.106 ms 2024-05-09T08:53:13.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 317984 29.71 MB/sec execute 448 sec latency 53.522 ms 2024-05-09T08:53:13.411 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:13 smithi160 ceph-mon[34070]: pgmap v1075: 97 pgs: 97 active+clean; 814 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 441 op/s 2024-05-09T08:53:13.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:13 smithi133 ceph-mon[29610]: pgmap v1075: 97 pgs: 97 active+clean; 814 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 441 op/s 2024-05-09T08:53:13.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:13 smithi188 ceph-mon[34645]: pgmap v1075: 97 pgs: 97 active+clean; 814 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 441 op/s 2024-05-09T08:53:14.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 318481 29.71 MB/sec execute 449 sec latency 92.829 ms 2024-05-09T08:53:15.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 318732 29.67 MB/sec execute 450 sec latency 546.518 ms 2024-05-09T08:53:15.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:15 smithi160 ceph-mon[34070]: pgmap v1076: 97 pgs: 97 active+clean; 815 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 406 op/s 2024-05-09T08:53:15.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:15 smithi133 ceph-mon[29610]: pgmap v1076: 97 pgs: 97 active+clean; 815 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 406 op/s 2024-05-09T08:53:15.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:15 smithi188 ceph-mon[34645]: pgmap v1076: 97 pgs: 97 active+clean; 815 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 406 op/s 2024-05-09T08:53:16.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 318827 29.61 MB/sec execute 451 sec latency 923.827 ms 2024-05-09T08:53:17.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 319379 29.59 MB/sec execute 452 sec latency 149.966 ms 2024-05-09T08:53:17.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:17 smithi160 ceph-mon[34070]: pgmap v1077: 97 pgs: 97 active+clean; 807 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-05-09T08:53:17.430 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:17 smithi133 ceph-mon[29610]: pgmap v1077: 97 pgs: 97 active+clean; 807 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-05-09T08:53:17.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:17 smithi188 ceph-mon[34645]: pgmap v1077: 97 pgs: 97 active+clean; 807 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.8 MiB/s wr, 512 op/s 2024-05-09T08:53:18.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 319936 29.60 MB/sec execute 453 sec latency 115.611 ms 2024-05-09T08:53:19.406 INFO:tasks.workunit.client.0.smithi133.stdout: 10 320522 29.62 MB/sec execute 454 sec latency 81.571 ms 2024-05-09T08:53:19.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:19 smithi160 ceph-mon[34070]: pgmap v1078: 97 pgs: 97 active+clean; 812 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 508 op/s 2024-05-09T08:53:19.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:19 smithi133 ceph-mon[29610]: pgmap v1078: 97 pgs: 97 active+clean; 812 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 508 op/s 2024-05-09T08:53:19.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:19 smithi188 ceph-mon[34645]: pgmap v1078: 97 pgs: 97 active+clean; 812 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.7 MiB/s wr, 508 op/s 2024-05-09T08:53:20.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 321046 29.61 MB/sec execute 455 sec latency 73.422 ms 2024-05-09T08:53:21.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 321299 29.58 MB/sec execute 456 sec latency 312.206 ms 2024-05-09T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:21 smithi160 ceph-mon[34070]: pgmap v1079: 97 pgs: 97 active+clean; 808 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 569 op/s 2024-05-09T08:53:21.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:21 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:53:21.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:21 smithi133 ceph-mon[29610]: pgmap v1079: 97 pgs: 97 active+clean; 808 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 569 op/s 2024-05-09T08:53:21.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:21 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:53:21.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:21 smithi188 ceph-mon[34645]: pgmap v1079: 97 pgs: 97 active+clean; 808 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 11 MiB/s wr, 569 op/s 2024-05-09T08:53:21.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:21 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:53:22.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 321647 29.55 MB/sec execute 457 sec latency 308.363 ms 2024-05-09T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:53:22.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:53:22.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:53:22.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:53:22.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:53:22.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:53:22.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:53:22.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:53:23.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 322173 29.56 MB/sec execute 458 sec latency 80.315 ms 2024-05-09T08:53:23.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:23 smithi160 ceph-mon[34070]: pgmap v1080: 97 pgs: 97 active+clean; 800 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-05-09T08:53:23.427 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:23 smithi133 ceph-mon[29610]: pgmap v1080: 97 pgs: 97 active+clean; 800 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-05-09T08:53:23.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:23 smithi188 ceph-mon[34645]: pgmap v1080: 97 pgs: 97 active+clean; 800 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-05-09T08:53:24.407 INFO:tasks.workunit.client.0.smithi133.stdout: 10 322656 29.55 MB/sec execute 459 sec latency 98.581 ms 2024-05-09T08:53:25.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 323230 29.54 MB/sec execute 460 sec latency 52.061 ms 2024-05-09T08:53:25.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:25 smithi160 ceph-mon[34070]: pgmap v1081: 97 pgs: 97 active+clean; 796 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-05-09T08:53:25.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:25 smithi133 ceph-mon[29610]: pgmap v1081: 97 pgs: 97 active+clean; 796 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-05-09T08:53:25.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:25 smithi188 ceph-mon[34645]: pgmap v1081: 97 pgs: 97 active+clean; 796 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-05-09T08:53:26.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 323753 29.54 MB/sec execute 461 sec latency 70.476 ms 2024-05-09T08:53:26.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:26 smithi160 ceph-mon[34070]: pgmap v1082: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-05-09T08:53:26.426 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:26 smithi133 ceph-mon[29610]: pgmap v1082: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-05-09T08:53:26.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:26 smithi188 ceph-mon[34645]: pgmap v1082: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 727 op/s 2024-05-09T08:53:27.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 324409 29.55 MB/sec execute 462 sec latency 69.726 ms 2024-05-09T08:53:28.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 325038 29.56 MB/sec execute 463 sec latency 89.931 ms 2024-05-09T08:53:28.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:28 smithi160 ceph-mon[34070]: pgmap v1083: 97 pgs: 97 active+clean; 794 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 659 op/s 2024-05-09T08:53:28.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:28 smithi133 ceph-mon[29610]: pgmap v1083: 97 pgs: 97 active+clean; 794 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 659 op/s 2024-05-09T08:53:28.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:28 smithi188 ceph-mon[34645]: pgmap v1083: 97 pgs: 97 active+clean; 794 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 14 MiB/s wr, 659 op/s 2024-05-09T08:53:29.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 325664 29.58 MB/sec execute 464 sec latency 54.615 ms 2024-05-09T08:53:30.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 326148 29.58 MB/sec execute 465 sec latency 91.716 ms 2024-05-09T08:53:30.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:30 smithi160 ceph-mon[34070]: pgmap v1084: 97 pgs: 97 active+clean; 793 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-05-09T08:53:30.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:30 smithi133 ceph-mon[29610]: pgmap v1084: 97 pgs: 97 active+clean; 793 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-05-09T08:53:30.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:30 smithi188 ceph-mon[34645]: pgmap v1084: 97 pgs: 97 active+clean; 793 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 11 MiB/s wr, 625 op/s 2024-05-09T08:53:31.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 326604 29.56 MB/sec execute 466 sec latency 87.858 ms 2024-05-09T08:53:32.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 327274 29.55 MB/sec execute 467 sec latency 57.721 ms 2024-05-09T08:53:32.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:32 smithi160 ceph-mon[34070]: pgmap v1085: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-05-09T08:53:32.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:32 smithi133 ceph-mon[29610]: pgmap v1085: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-05-09T08:53:32.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:32 smithi188 ceph-mon[34645]: pgmap v1085: 97 pgs: 97 active+clean; 802 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 9.1 MiB/s wr, 636 op/s 2024-05-09T08:53:33.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 327827 29.56 MB/sec execute 468 sec latency 78.493 ms 2024-05-09T08:53:34.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 328446 29.57 MB/sec execute 469 sec latency 131.304 ms 2024-05-09T08:53:34.663 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:34 smithi160 ceph-mon[34070]: pgmap v1086: 97 pgs: 97 active+clean; 803 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 490 op/s 2024-05-09T08:53:34.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:34 smithi133 ceph-mon[29610]: pgmap v1086: 97 pgs: 97 active+clean; 803 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 490 op/s 2024-05-09T08:53:34.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:34 smithi188 ceph-mon[34645]: pgmap v1086: 97 pgs: 97 active+clean; 803 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 6.6 MiB/s wr, 490 op/s 2024-05-09T08:53:35.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 329091 29.59 MB/sec execute 470 sec latency 92.580 ms 2024-05-09T08:53:36.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 329718 29.60 MB/sec execute 471 sec latency 96.132 ms 2024-05-09T08:53:36.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:36 smithi160 ceph-mon[34070]: pgmap v1087: 97 pgs: 97 active+clean; 808 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 499 op/s 2024-05-09T08:53:36.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:36 smithi133 ceph-mon[29610]: pgmap v1087: 97 pgs: 97 active+clean; 808 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 499 op/s 2024-05-09T08:53:36.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:36 smithi188 ceph-mon[34645]: pgmap v1087: 97 pgs: 97 active+clean; 808 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.6 MiB/s wr, 499 op/s 2024-05-09T08:53:37.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 330217 29.60 MB/sec execute 472 sec latency 112.192 ms 2024-05-09T08:53:38.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 330762 29.61 MB/sec execute 473 sec latency 88.151 ms 2024-05-09T08:53:38.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:38 smithi160 ceph-mon[34070]: pgmap v1088: 97 pgs: 97 active+clean; 807 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-05-09T08:53:38.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:38 smithi133 ceph-mon[29610]: pgmap v1088: 97 pgs: 97 active+clean; 807 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-05-09T08:53:38.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:38 smithi188 ceph-mon[34645]: pgmap v1088: 97 pgs: 97 active+clean; 807 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-05-09T08:53:39.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 331271 29.57 MB/sec execute 474 sec latency 70.612 ms 2024-05-09T08:53:40.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 331788 29.57 MB/sec execute 475 sec latency 202.859 ms 2024-05-09T08:53:40.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:40 smithi160 ceph-mon[34070]: pgmap v1089: 97 pgs: 97 active+clean; 794 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:53:40.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:40 smithi133 ceph-mon[29610]: pgmap v1089: 97 pgs: 97 active+clean; 794 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:53:40.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:40 smithi188 ceph-mon[34645]: pgmap v1089: 97 pgs: 97 active+clean; 794 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 379 op/s 2024-05-09T08:53:41.408 INFO:tasks.workunit.client.0.smithi133.stdout: 10 332565 29.60 MB/sec execute 476 sec latency 219.772 ms 2024-05-09T08:53:42.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 333316 29.63 MB/sec execute 477 sec latency 60.866 ms 2024-05-09T08:53:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:42 smithi160 ceph-mon[34070]: pgmap v1090: 97 pgs: 97 active+clean; 803 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-05-09T08:53:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:42 smithi133 ceph-mon[29610]: pgmap v1090: 97 pgs: 97 active+clean; 803 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-05-09T08:53:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:42 smithi188 ceph-mon[34645]: pgmap v1090: 97 pgs: 97 active+clean; 803 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-05-09T08:53:43.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 333931 29.65 MB/sec execute 478 sec latency 61.445 ms 2024-05-09T08:53:44.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 334346 29.63 MB/sec execute 479 sec latency 118.225 ms 2024-05-09T08:53:44.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:44 smithi160 ceph-mon[34070]: pgmap v1091: 97 pgs: 97 active+clean; 806 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-05-09T08:53:44.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:44 smithi133 ceph-mon[29610]: pgmap v1091: 97 pgs: 97 active+clean; 806 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-05-09T08:53:44.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:44 smithi188 ceph-mon[34645]: pgmap v1091: 97 pgs: 97 active+clean; 806 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-05-09T08:53:45.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 334930 29.62 MB/sec execute 480 sec latency 93.574 ms 2024-05-09T08:53:46.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 335517 29.62 MB/sec execute 481 sec latency 50.035 ms 2024-05-09T08:53:46.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:46 smithi160 ceph-mon[34070]: pgmap v1092: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-05-09T08:53:46.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:46 smithi133 ceph-mon[29610]: pgmap v1092: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-05-09T08:53:46.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:46 smithi188 ceph-mon[34645]: pgmap v1092: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-05-09T08:53:47.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 336169 29.64 MB/sec execute 482 sec latency 53.845 ms 2024-05-09T08:53:48.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 336800 29.65 MB/sec execute 483 sec latency 49.686 ms 2024-05-09T08:53:48.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:48 smithi160 ceph-mon[34070]: pgmap v1093: 97 pgs: 97 active+clean; 817 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:53:48.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:48 smithi133 ceph-mon[29610]: pgmap v1093: 97 pgs: 97 active+clean; 817 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:53:48.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:48 smithi188 ceph-mon[34645]: pgmap v1093: 97 pgs: 97 active+clean; 817 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:53:49.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 337492 29.67 MB/sec execute 484 sec latency 74.598 ms 2024-05-09T08:53:50.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 338061 29.67 MB/sec execute 485 sec latency 92.333 ms 2024-05-09T08:53:50.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:50 smithi160 ceph-mon[34070]: pgmap v1094: 97 pgs: 97 active+clean; 816 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-05-09T08:53:50.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:50 smithi133 ceph-mon[29610]: pgmap v1094: 97 pgs: 97 active+clean; 816 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-05-09T08:53:50.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:50 smithi188 ceph-mon[34645]: pgmap v1094: 97 pgs: 97 active+clean; 816 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-05-09T08:53:51.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 338543 29.68 MB/sec execute 486 sec latency 70.992 ms 2024-05-09T08:53:52.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 339030 29.66 MB/sec execute 487 sec latency 81.229 ms 2024-05-09T08:53:52.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:52 smithi160 ceph-mon[34070]: pgmap v1095: 97 pgs: 97 active+clean; 815 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-05-09T08:53:52.676 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:52 smithi133 ceph-mon[29610]: pgmap v1095: 97 pgs: 97 active+clean; 815 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-05-09T08:53:52.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:52 smithi188 ceph-mon[34645]: pgmap v1095: 97 pgs: 97 active+clean; 815 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.5 MiB/s wr, 413 op/s 2024-05-09T08:53:53.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 339650 29.67 MB/sec execute 488 sec latency 45.470 ms 2024-05-09T08:53:54.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 340245 29.66 MB/sec execute 489 sec latency 51.204 ms 2024-05-09T08:53:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:54 smithi160 ceph-mon[34070]: pgmap v1096: 97 pgs: 97 active+clean; 821 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:53:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:54 smithi133 ceph-mon[29610]: pgmap v1096: 97 pgs: 97 active+clean; 821 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:53:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:54 smithi188 ceph-mon[34645]: pgmap v1096: 97 pgs: 97 active+clean; 821 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-05-09T08:53:55.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 340989 29.68 MB/sec execute 490 sec latency 226.683 ms 2024-05-09T08:53:56.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 341619 29.71 MB/sec execute 491 sec latency 59.304 ms 2024-05-09T08:53:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:56 smithi160 ceph-mon[34070]: pgmap v1097: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-05-09T08:53:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:56 smithi188 ceph-mon[34645]: pgmap v1097: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-05-09T08:53:56.936 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:56 smithi133 ceph-mon[29610]: pgmap v1097: 97 pgs: 97 active+clean; 809 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.6 MiB/s wr, 392 op/s 2024-05-09T08:53:57.409 INFO:tasks.workunit.client.0.smithi133.stdout: 10 341859 29.68 MB/sec execute 492 sec latency 548.777 ms 2024-05-09T08:53:58.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 341874 29.62 MB/sec execute 493 sec latency 1232.596 ms 2024-05-09T08:53:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:53:58 smithi160 ceph-mon[34070]: pgmap v1098: 97 pgs: 97 active+clean; 812 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-09T08:53:58.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:53:58 smithi133 ceph-mon[29610]: pgmap v1098: 97 pgs: 97 active+clean; 812 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-09T08:53:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:53:58 smithi188 ceph-mon[34645]: pgmap v1098: 97 pgs: 97 active+clean; 812 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-09T08:53:59.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 342023 29.57 MB/sec execute 494 sec latency 1419.975 ms 2024-05-09T08:54:00.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 342267 29.53 MB/sec execute 495 sec latency 168.954 ms 2024-05-09T08:54:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:00 smithi160 ceph-mon[34070]: pgmap v1099: 97 pgs: 97 active+clean; 827 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.9 MiB/s wr, 478 op/s 2024-05-09T08:54:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:00 smithi133 ceph-mon[29610]: pgmap v1099: 97 pgs: 97 active+clean; 827 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.9 MiB/s wr, 478 op/s 2024-05-09T08:54:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:00 smithi188 ceph-mon[34645]: pgmap v1099: 97 pgs: 97 active+clean; 827 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.9 MiB/s wr, 478 op/s 2024-05-09T08:54:01.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 342636 29.51 MB/sec execute 496 sec latency 151.856 ms 2024-05-09T08:54:02.410 INFO:tasks.workunit.client.0.smithi133.stdout: 10 343151 29.50 MB/sec execute 497 sec latency 80.904 ms 2024-05-09T08:54:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:02 smithi160 ceph-mon[34070]: pgmap v1100: 97 pgs: 97 active+clean; 829 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 583 op/s 2024-05-09T08:54:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:02 smithi133 ceph-mon[29610]: pgmap v1100: 97 pgs: 97 active+clean; 829 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 583 op/s 2024-05-09T08:54:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:02 smithi188 ceph-mon[34645]: pgmap v1100: 97 pgs: 97 active+clean; 829 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 583 op/s 2024-05-09T08:54:03.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 343342 29.46 MB/sec execute 498 sec latency 496.697 ms 2024-05-09T08:54:04.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 343793 29.45 MB/sec execute 499 sec latency 117.530 ms 2024-05-09T08:54:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:04 smithi160 ceph-mon[34070]: pgmap v1101: 97 pgs: 97 active+clean; 817 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-05-09T08:54:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:04 smithi133 ceph-mon[29610]: pgmap v1101: 97 pgs: 97 active+clean; 817 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-05-09T08:54:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:04 smithi188 ceph-mon[34645]: pgmap v1101: 97 pgs: 97 active+clean; 817 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 638 op/s 2024-05-09T08:54:05.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 344347 29.46 MB/sec execute 500 sec latency 83.468 ms 2024-05-09T08:54:06.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 344999 29.47 MB/sec execute 501 sec latency 56.107 ms 2024-05-09T08:54:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:06 smithi160 ceph-mon[34070]: pgmap v1102: 97 pgs: 97 active+clean; 788 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-05-09T08:54:06.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:06 smithi133 ceph-mon[29610]: pgmap v1102: 97 pgs: 97 active+clean; 788 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-05-09T08:54:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:06 smithi188 ceph-mon[34645]: pgmap v1102: 97 pgs: 97 active+clean; 788 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 16 MiB/s wr, 694 op/s 2024-05-09T08:54:07.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 345582 29.49 MB/sec execute 502 sec latency 47.236 ms 2024-05-09T08:54:08.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 346059 29.47 MB/sec execute 503 sec latency 91.601 ms 2024-05-09T08:54:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:08 smithi160 ceph-mon[34070]: pgmap v1103: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-09T08:54:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:08 smithi133 ceph-mon[29610]: pgmap v1103: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-09T08:54:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:08 smithi188 ceph-mon[34645]: pgmap v1103: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 17 MiB/s wr, 787 op/s 2024-05-09T08:54:09.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 346639 29.46 MB/sec execute 504 sec latency 97.078 ms 2024-05-09T08:54:10.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 347285 29.49 MB/sec execute 505 sec latency 81.102 ms 2024-05-09T08:54:10.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:10 smithi160 ceph-mon[34070]: pgmap v1104: 97 pgs: 97 active+clean; 784 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 781 op/s 2024-05-09T08:54:10.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:10 smithi133 ceph-mon[29610]: pgmap v1104: 97 pgs: 97 active+clean; 784 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 781 op/s 2024-05-09T08:54:10.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:10 smithi188 ceph-mon[34645]: pgmap v1104: 97 pgs: 97 active+clean; 784 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 17 MiB/s wr, 781 op/s 2024-05-09T08:54:11.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 347864 29.48 MB/sec execute 506 sec latency 114.748 ms 2024-05-09T08:54:12.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 348541 29.49 MB/sec execute 507 sec latency 53.325 ms 2024-05-09T08:54:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:12 smithi160 ceph-mon[34070]: pgmap v1105: 97 pgs: 97 active+clean; 790 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:54:12.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:12 smithi133 ceph-mon[29610]: pgmap v1105: 97 pgs: 97 active+clean; 790 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:54:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:12 smithi188 ceph-mon[34645]: pgmap v1105: 97 pgs: 97 active+clean; 790 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 724 op/s 2024-05-09T08:54:13.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 349035 29.52 MB/sec execute 508 sec latency 127.307 ms 2024-05-09T08:54:14.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 349573 29.54 MB/sec execute 509 sec latency 54.332 ms 2024-05-09T08:54:14.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:14 smithi160 ceph-mon[34070]: pgmap v1106: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 MiB/s wr, 547 op/s 2024-05-09T08:54:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:14 smithi133 ceph-mon[29610]: pgmap v1106: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 MiB/s wr, 547 op/s 2024-05-09T08:54:14.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:14 smithi188 ceph-mon[34645]: pgmap v1106: 97 pgs: 97 active+clean; 791 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.4 MiB/s wr, 547 op/s 2024-05-09T08:54:15.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 350096 29.52 MB/sec execute 510 sec latency 91.257 ms 2024-05-09T08:54:16.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 350638 29.51 MB/sec execute 511 sec latency 103.442 ms 2024-05-09T08:54:16.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:16 smithi160 ceph-mon[34070]: pgmap v1107: 97 pgs: 97 active+clean; 794 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 538 op/s 2024-05-09T08:54:16.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:16 smithi133 ceph-mon[29610]: pgmap v1107: 97 pgs: 97 active+clean; 794 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 538 op/s 2024-05-09T08:54:16.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:16 smithi188 ceph-mon[34645]: pgmap v1107: 97 pgs: 97 active+clean; 794 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 538 op/s 2024-05-09T08:54:17.411 INFO:tasks.workunit.client.0.smithi133.stdout: 10 351370 29.54 MB/sec execute 512 sec latency 67.300 ms 2024-05-09T08:54:18.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 351927 29.54 MB/sec execute 513 sec latency 154.018 ms 2024-05-09T08:54:18.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:18 smithi160 ceph-mon[34070]: pgmap v1108: 97 pgs: 97 active+clean; 800 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 493 op/s 2024-05-09T08:54:18.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:18 smithi133 ceph-mon[29610]: pgmap v1108: 97 pgs: 97 active+clean; 800 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 493 op/s 2024-05-09T08:54:18.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:18 smithi188 ceph-mon[34645]: pgmap v1108: 97 pgs: 97 active+clean; 800 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 493 op/s 2024-05-09T08:54:19.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 352307 29.52 MB/sec execute 514 sec latency 401.922 ms 2024-05-09T08:54:20.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 352838 29.52 MB/sec execute 515 sec latency 293.472 ms 2024-05-09T08:54:20.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:20 smithi160 ceph-mon[34070]: pgmap v1109: 97 pgs: 97 active+clean; 808 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-05-09T08:54:20.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:20 smithi133 ceph-mon[29610]: pgmap v1109: 97 pgs: 97 active+clean; 808 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-05-09T08:54:20.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:20 smithi188 ceph-mon[34645]: pgmap v1109: 97 pgs: 97 active+clean; 808 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-05-09T08:54:21.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 353372 29.54 MB/sec execute 516 sec latency 101.750 ms 2024-05-09T08:54:22.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 353797 29.52 MB/sec execute 517 sec latency 106.867 ms 2024-05-09T08:54:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:22 smithi160 ceph-mon[34070]: pgmap v1110: 97 pgs: 97 active+clean; 810 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-05-09T08:54:22.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:22 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:54:22.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:22 smithi133 ceph-mon[29610]: pgmap v1110: 97 pgs: 97 active+clean; 810 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-05-09T08:54:22.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:22 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:54:22.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:22 smithi188 ceph-mon[34645]: pgmap v1110: 97 pgs: 97 active+clean; 810 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.3 MiB/s wr, 451 op/s 2024-05-09T08:54:22.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:22 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:54:23.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 353928 29.47 MB/sec execute 518 sec latency 813.420 ms 2024-05-09T08:54:24.412 INFO:tasks.workunit.client.0.smithi133.stdout: 10 354186 29.44 MB/sec execute 519 sec latency 1074.266 ms 2024-05-09T08:54:24.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:24 smithi160 ceph-mon[34070]: pgmap v1111: 97 pgs: 97 active+clean; 803 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-09T08:54:24.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:24 smithi133 ceph-mon[29610]: pgmap v1111: 97 pgs: 97 active+clean; 803 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-09T08:54:24.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:24 smithi188 ceph-mon[34645]: pgmap v1111: 97 pgs: 97 active+clean; 803 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 387 op/s 2024-05-09T08:54:25.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 354581 29.41 MB/sec execute 520 sec latency 132.563 ms 2024-05-09T08:54:25.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:54:25.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:54:25.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:54:25.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:54:25.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:54:25.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:54:25.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:54:25.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:54:25.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:54:26.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 355272 29.43 MB/sec execute 521 sec latency 84.614 ms 2024-05-09T08:54:26.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:26 smithi160 ceph-mon[34070]: pgmap v1112: 97 pgs: 97 active+clean; 823 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 529 op/s 2024-05-09T08:54:26.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:26 smithi133 ceph-mon[29610]: pgmap v1112: 97 pgs: 97 active+clean; 823 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 529 op/s 2024-05-09T08:54:26.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:26 smithi188 ceph-mon[34645]: pgmap v1112: 97 pgs: 97 active+clean; 823 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 529 op/s 2024-05-09T08:54:27.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 356017 29.44 MB/sec execute 522 sec latency 70.249 ms 2024-05-09T08:54:28.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 356755 29.48 MB/sec execute 523 sec latency 62.743 ms 2024-05-09T08:54:28.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:28 smithi160 ceph-mon[34070]: pgmap v1113: 97 pgs: 97 active+clean; 825 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-09T08:54:28.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:28 smithi133 ceph-mon[29610]: pgmap v1113: 97 pgs: 97 active+clean; 825 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-09T08:54:28.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:28 smithi188 ceph-mon[34645]: pgmap v1113: 97 pgs: 97 active+clean; 825 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-05-09T08:54:29.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 357184 29.48 MB/sec execute 524 sec latency 73.312 ms 2024-05-09T08:54:30.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 357573 29.46 MB/sec execute 525 sec latency 76.479 ms 2024-05-09T08:54:30.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:30 smithi160 ceph-mon[34070]: pgmap v1114: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 622 op/s 2024-05-09T08:54:30.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:30 smithi133 ceph-mon[29610]: pgmap v1114: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 622 op/s 2024-05-09T08:54:30.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:30 smithi188 ceph-mon[34645]: pgmap v1114: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 14 MiB/s wr, 622 op/s 2024-05-09T08:54:31.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 358053 29.44 MB/sec execute 526 sec latency 76.874 ms 2024-05-09T08:54:32.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 358607 29.43 MB/sec execute 527 sec latency 84.842 ms 2024-05-09T08:54:32.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:32 smithi160 ceph-mon[34070]: pgmap v1115: 97 pgs: 97 active+clean; 796 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 13 MiB/s wr, 636 op/s 2024-05-09T08:54:32.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:32 smithi133 ceph-mon[29610]: pgmap v1115: 97 pgs: 97 active+clean; 796 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 13 MiB/s wr, 636 op/s 2024-05-09T08:54:32.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:32 smithi188 ceph-mon[34645]: pgmap v1115: 97 pgs: 97 active+clean; 796 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 13 MiB/s wr, 636 op/s 2024-05-09T08:54:33.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 359294 29.45 MB/sec execute 528 sec latency 104.747 ms 2024-05-09T08:54:34.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 359849 29.44 MB/sec execute 529 sec latency 105.997 ms 2024-05-09T08:54:34.913 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:34 smithi160 ceph-mon[34070]: pgmap v1116: 97 pgs: 97 active+clean; 801 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-05-09T08:54:34.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:34 smithi133 ceph-mon[29610]: pgmap v1116: 97 pgs: 97 active+clean; 801 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-05-09T08:54:34.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:34 smithi188 ceph-mon[34645]: pgmap v1116: 97 pgs: 97 active+clean; 801 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-05-09T08:54:35.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 360587 29.47 MB/sec execute 530 sec latency 69.347 ms 2024-05-09T08:54:36.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 361236 29.50 MB/sec execute 531 sec latency 59.513 ms 2024-05-09T08:54:36.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:36 smithi160 ceph-mon[34070]: pgmap v1117: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-05-09T08:54:36.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:36 smithi133 ceph-mon[29610]: pgmap v1117: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-05-09T08:54:36.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:36 smithi188 ceph-mon[34645]: pgmap v1117: 97 pgs: 97 active+clean; 804 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 592 op/s 2024-05-09T08:54:37.413 INFO:tasks.workunit.client.0.smithi133.stdout: 10 361632 29.49 MB/sec execute 532 sec latency 57.559 ms 2024-05-09T08:54:38.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 362022 29.46 MB/sec execute 533 sec latency 72.306 ms 2024-05-09T08:54:38.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:38 smithi160 ceph-mon[34070]: pgmap v1118: 97 pgs: 97 active+clean; 812 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.4 MiB/s wr, 535 op/s 2024-05-09T08:54:38.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:38 smithi133 ceph-mon[29610]: pgmap v1118: 97 pgs: 97 active+clean; 812 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.4 MiB/s wr, 535 op/s 2024-05-09T08:54:38.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:38 smithi188 ceph-mon[34645]: pgmap v1118: 97 pgs: 97 active+clean; 812 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 7.4 MiB/s wr, 535 op/s 2024-05-09T08:54:39.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 362463 29.44 MB/sec execute 534 sec latency 86.526 ms 2024-05-09T08:54:40.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 363148 29.45 MB/sec execute 535 sec latency 79.706 ms 2024-05-09T08:54:40.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:40 smithi160 ceph-mon[34070]: pgmap v1119: 97 pgs: 97 active+clean; 820 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-05-09T08:54:40.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:40 smithi133 ceph-mon[29610]: pgmap v1119: 97 pgs: 97 active+clean; 820 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-05-09T08:54:40.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:40 smithi188 ceph-mon[34645]: pgmap v1119: 97 pgs: 97 active+clean; 820 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-05-09T08:54:41.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 363882 29.46 MB/sec execute 536 sec latency 65.935 ms 2024-05-09T08:54:42.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 364658 29.50 MB/sec execute 537 sec latency 59.657 ms 2024-05-09T08:54:42.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:42 smithi160 ceph-mon[34070]: pgmap v1120: 97 pgs: 97 active+clean; 822 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:54:42.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:42 smithi133 ceph-mon[29610]: pgmap v1120: 97 pgs: 97 active+clean; 822 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:54:42.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:42 smithi188 ceph-mon[34645]: pgmap v1120: 97 pgs: 97 active+clean; 822 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:54:43.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 365261 29.52 MB/sec execute 538 sec latency 54.363 ms 2024-05-09T08:54:44.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 365456 29.49 MB/sec execute 539 sec latency 449.813 ms 2024-05-09T08:54:44.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:44 smithi160 ceph-mon[34070]: pgmap v1121: 97 pgs: 97 active+clean; 826 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-09T08:54:44.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:44 smithi133 ceph-mon[29610]: pgmap v1121: 97 pgs: 97 active+clean; 826 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-09T08:54:44.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:44 smithi188 ceph-mon[34645]: pgmap v1121: 97 pgs: 97 active+clean; 826 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.2 MiB/s wr, 377 op/s 2024-05-09T08:54:45.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 365800 29.46 MB/sec execute 540 sec latency 168.264 ms 2024-05-09T08:54:46.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 366219 29.43 MB/sec execute 541 sec latency 125.953 ms 2024-05-09T08:54:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:46 smithi160 ceph-mon[34070]: pgmap v1122: 97 pgs: 97 active+clean; 836 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-05-09T08:54:46.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:46 smithi133 ceph-mon[29610]: pgmap v1122: 97 pgs: 97 active+clean; 836 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-05-09T08:54:46.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:46 smithi188 ceph-mon[34645]: pgmap v1122: 97 pgs: 97 active+clean; 836 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-05-09T08:54:47.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 366757 29.44 MB/sec execute 542 sec latency 108.422 ms 2024-05-09T08:54:48.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 367441 29.44 MB/sec execute 543 sec latency 56.610 ms 2024-05-09T08:54:48.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:48 smithi160 ceph-mon[34070]: pgmap v1123: 97 pgs: 97 active+clean; 839 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 414 op/s 2024-05-09T08:54:48.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:48 smithi133 ceph-mon[29610]: pgmap v1123: 97 pgs: 97 active+clean; 839 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 414 op/s 2024-05-09T08:54:48.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:48 smithi188 ceph-mon[34645]: pgmap v1123: 97 pgs: 97 active+clean; 839 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 414 op/s 2024-05-09T08:54:49.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 367951 29.42 MB/sec execute 544 sec latency 364.852 ms 2024-05-09T08:54:50.414 INFO:tasks.workunit.client.0.smithi133.stdout: 10 368061 29.38 MB/sec execute 545 sec latency 757.214 ms 2024-05-09T08:54:50.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:50 smithi160 ceph-mon[34070]: pgmap v1124: 97 pgs: 97 active+clean; 838 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 456 op/s 2024-05-09T08:54:50.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:50 smithi133 ceph-mon[29610]: pgmap v1124: 97 pgs: 97 active+clean; 838 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 456 op/s 2024-05-09T08:54:50.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:50 smithi188 ceph-mon[34645]: pgmap v1124: 97 pgs: 97 active+clean; 838 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.7 MiB/s wr, 456 op/s 2024-05-09T08:54:51.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 368403 29.37 MB/sec execute 546 sec latency 338.434 ms 2024-05-09T08:54:52.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 368977 29.39 MB/sec execute 547 sec latency 119.741 ms 2024-05-09T08:54:52.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:52 smithi160 ceph-mon[34070]: pgmap v1125: 97 pgs: 97 active+clean; 829 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2024-05-09T08:54:52.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:52 smithi133 ceph-mon[29610]: pgmap v1125: 97 pgs: 97 active+clean; 829 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2024-05-09T08:54:52.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:52 smithi188 ceph-mon[34645]: pgmap v1125: 97 pgs: 97 active+clean; 829 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.7 MiB/s wr, 451 op/s 2024-05-09T08:54:53.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 369533 29.40 MB/sec execute 548 sec latency 84.386 ms 2024-05-09T08:54:54.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 369953 29.39 MB/sec execute 549 sec latency 127.174 ms 2024-05-09T08:54:54.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:54 smithi160 ceph-mon[34070]: pgmap v1126: 97 pgs: 97 active+clean; 828 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-05-09T08:54:54.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:54 smithi133 ceph-mon[29610]: pgmap v1126: 97 pgs: 97 active+clean; 828 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-05-09T08:54:54.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:54 smithi188 ceph-mon[34645]: pgmap v1126: 97 pgs: 97 active+clean; 828 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 498 op/s 2024-05-09T08:54:55.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 370327 29.36 MB/sec execute 550 sec latency 102.397 ms 2024-05-09T08:54:56.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 370744 29.35 MB/sec execute 551 sec latency 117.238 ms 2024-05-09T08:54:56.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:56 smithi160 ceph-mon[34070]: pgmap v1127: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 584 op/s 2024-05-09T08:54:56.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:56 smithi133 ceph-mon[29610]: pgmap v1127: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 584 op/s 2024-05-09T08:54:56.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:56 smithi188 ceph-mon[34645]: pgmap v1127: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 11 MiB/s wr, 584 op/s 2024-05-09T08:54:57.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 371279 29.33 MB/sec execute 552 sec latency 67.817 ms 2024-05-09T08:54:58.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 371854 29.34 MB/sec execute 553 sec latency 91.845 ms 2024-05-09T08:54:58.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:54:58 smithi160 ceph-mon[34070]: pgmap v1128: 97 pgs: 97 active+clean; 792 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:54:58.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:54:58 smithi133 ceph-mon[29610]: pgmap v1128: 97 pgs: 97 active+clean; 792 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:54:58.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:54:58 smithi188 ceph-mon[34645]: pgmap v1128: 97 pgs: 97 active+clean; 792 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 634 op/s 2024-05-09T08:54:59.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 372501 29.35 MB/sec execute 554 sec latency 65.224 ms 2024-05-09T08:55:00.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 373078 29.37 MB/sec execute 555 sec latency 65.556 ms 2024-05-09T08:55:00.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:00 smithi160 ceph-mon[34070]: pgmap v1129: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-05-09T08:55:00.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:00 smithi133 ceph-mon[29610]: pgmap v1129: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-05-09T08:55:00.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:00 smithi188 ceph-mon[34645]: pgmap v1129: 97 pgs: 97 active+clean; 794 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 582 op/s 2024-05-09T08:55:01.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 373562 29.36 MB/sec execute 556 sec latency 115.593 ms 2024-05-09T08:55:02.415 INFO:tasks.workunit.client.0.smithi133.stdout: 10 374077 29.36 MB/sec execute 557 sec latency 87.297 ms 2024-05-09T08:55:02.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:02 smithi160 ceph-mon[34070]: pgmap v1130: 97 pgs: 97 active+clean; 797 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 484 op/s 2024-05-09T08:55:02.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:02 smithi133 ceph-mon[29610]: pgmap v1130: 97 pgs: 97 active+clean; 797 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 484 op/s 2024-05-09T08:55:02.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:02 smithi188 ceph-mon[34645]: pgmap v1130: 97 pgs: 97 active+clean; 797 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 484 op/s 2024-05-09T08:55:03.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 374562 29.36 MB/sec execute 558 sec latency 90.223 ms 2024-05-09T08:55:04.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 375032 29.34 MB/sec execute 559 sec latency 81.482 ms 2024-05-09T08:55:04.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:04 smithi160 ceph-mon[34070]: pgmap v1131: 97 pgs: 97 active+clean; 805 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 493 op/s 2024-05-09T08:55:04.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:04 smithi133 ceph-mon[29610]: pgmap v1131: 97 pgs: 97 active+clean; 805 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 493 op/s 2024-05-09T08:55:04.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:04 smithi188 ceph-mon[34645]: pgmap v1131: 97 pgs: 97 active+clean; 805 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.6 MiB/s wr, 493 op/s 2024-05-09T08:55:05.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 375566 29.33 MB/sec execute 560 sec latency 92.400 ms 2024-05-09T08:55:06.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 376255 29.34 MB/sec execute 561 sec latency 39.696 ms 2024-05-09T08:55:06.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:06 smithi160 ceph-mon[34070]: pgmap v1132: 97 pgs: 97 active+clean; 807 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 440 op/s 2024-05-09T08:55:06.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:06 smithi133 ceph-mon[29610]: pgmap v1132: 97 pgs: 97 active+clean; 807 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 440 op/s 2024-05-09T08:55:06.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:06 smithi188 ceph-mon[34645]: pgmap v1132: 97 pgs: 97 active+clean; 807 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.2 MiB/s wr, 440 op/s 2024-05-09T08:55:07.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 376958 29.36 MB/sec execute 562 sec latency 56.192 ms 2024-05-09T08:55:08.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 377447 29.36 MB/sec execute 563 sec latency 96.608 ms 2024-05-09T08:55:08.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:08 smithi160 ceph-mon[34070]: pgmap v1133: 97 pgs: 97 active+clean; 814 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-05-09T08:55:08.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:08 smithi133 ceph-mon[29610]: pgmap v1133: 97 pgs: 97 active+clean; 814 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-05-09T08:55:08.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:08 smithi188 ceph-mon[34645]: pgmap v1133: 97 pgs: 97 active+clean; 814 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 400 op/s 2024-05-09T08:55:09.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 377983 29.37 MB/sec execute 564 sec latency 61.595 ms 2024-05-09T08:55:10.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 378486 29.36 MB/sec execute 565 sec latency 111.278 ms 2024-05-09T08:55:10.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:10 smithi160 ceph-mon[34070]: pgmap v1134: 97 pgs: 97 active+clean; 820 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-09T08:55:10.928 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:10 smithi133 ceph-mon[29610]: pgmap v1134: 97 pgs: 97 active+clean; 820 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-09T08:55:10.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:10 smithi188 ceph-mon[34645]: pgmap v1134: 97 pgs: 97 active+clean; 820 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-05-09T08:55:11.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 378920 29.35 MB/sec execute 566 sec latency 75.993 ms 2024-05-09T08:55:12.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 379466 29.35 MB/sec execute 567 sec latency 77.476 ms 2024-05-09T08:55:12.926 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:12 smithi133 ceph-mon[29610]: pgmap v1135: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-09T08:55:12.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:12 smithi188 ceph-mon[34645]: pgmap v1135: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-09T08:55:13.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:12 smithi160 ceph-mon[34070]: pgmap v1135: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 373 op/s 2024-05-09T08:55:13.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 380012 29.34 MB/sec execute 568 sec latency 85.057 ms 2024-05-09T08:55:14.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 380675 29.35 MB/sec execute 569 sec latency 45.997 ms 2024-05-09T08:55:14.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:14 smithi133 ceph-mon[29610]: pgmap v1136: 97 pgs: 97 active+clean; 821 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-09T08:55:14.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:14 smithi188 ceph-mon[34645]: pgmap v1136: 97 pgs: 97 active+clean; 821 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-09T08:55:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:14 smithi160 ceph-mon[34070]: pgmap v1136: 97 pgs: 97 active+clean; 821 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 347 op/s 2024-05-09T08:55:15.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 381118 29.34 MB/sec execute 570 sec latency 270.396 ms 2024-05-09T08:55:16.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 381378 29.32 MB/sec execute 571 sec latency 649.477 ms 2024-05-09T08:55:16.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:16 smithi188 ceph-mon[34645]: pgmap v1137: 97 pgs: 97 active+clean; 817 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-05-09T08:55:17.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:16 smithi160 ceph-mon[34070]: pgmap v1137: 97 pgs: 97 active+clean; 817 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-05-09T08:55:17.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:16 smithi133 ceph-mon[29610]: pgmap v1137: 97 pgs: 97 active+clean; 817 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 344 op/s 2024-05-09T08:55:17.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 381858 29.32 MB/sec execute 572 sec latency 123.483 ms 2024-05-09T08:55:18.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 382514 29.33 MB/sec execute 573 sec latency 53.122 ms 2024-05-09T08:55:18.932 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:18 smithi188 ceph-mon[34645]: pgmap v1138: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 395 op/s 2024-05-09T08:55:19.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:18 smithi160 ceph-mon[34070]: pgmap v1138: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 395 op/s 2024-05-09T08:55:19.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:18 smithi133 ceph-mon[29610]: pgmap v1138: 97 pgs: 97 active+clean; 820 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.1 MiB/s wr, 395 op/s 2024-05-09T08:55:19.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 383017 29.33 MB/sec execute 574 sec latency 66.160 ms 2024-05-09T08:55:20.416 INFO:tasks.workunit.client.0.smithi133.stdout: 10 383443 29.32 MB/sec execute 575 sec latency 80.438 ms 2024-05-09T08:55:21.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:20 smithi160 ceph-mon[34070]: pgmap v1139: 97 pgs: 97 active+clean; 819 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-05-09T08:55:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:20 smithi133 ceph-mon[29610]: pgmap v1139: 97 pgs: 97 active+clean; 819 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-05-09T08:55:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:20 smithi188 ceph-mon[34645]: pgmap v1139: 97 pgs: 97 active+clean; 819 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.6 MiB/s wr, 396 op/s 2024-05-09T08:55:21.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 383613 29.29 MB/sec execute 576 sec latency 942.100 ms 2024-05-09T08:55:22.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 383656 29.24 MB/sec execute 577 sec latency 1103.638 ms 2024-05-09T08:55:22.964 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:22 smithi160 ceph-mon[34070]: pgmap v1140: 97 pgs: 97 active+clean; 826 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 472 op/s 2024-05-09T08:55:22.981 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:22 smithi133 ceph-mon[29610]: pgmap v1140: 97 pgs: 97 active+clean; 826 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 472 op/s 2024-05-09T08:55:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:22 smithi188 ceph-mon[34645]: pgmap v1140: 97 pgs: 97 active+clean; 826 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.2 MiB/s wr, 472 op/s 2024-05-09T08:55:23.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 383971 29.21 MB/sec execute 578 sec latency 514.086 ms 2024-05-09T08:55:24.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 384452 29.19 MB/sec execute 579 sec latency 102.535 ms 2024-05-09T08:55:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:24 smithi160 ceph-mon[34070]: pgmap v1141: 97 pgs: 97 active+clean; 836 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 496 op/s 2024-05-09T08:55:25.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:24 smithi133 ceph-mon[29610]: pgmap v1141: 97 pgs: 97 active+clean; 836 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 496 op/s 2024-05-09T08:55:25.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:24 smithi188 ceph-mon[34645]: pgmap v1141: 97 pgs: 97 active+clean; 836 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 8.9 MiB/s wr, 496 op/s 2024-05-09T08:55:25.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 385013 29.21 MB/sec execute 580 sec latency 69.328 ms 2024-05-09T08:55:26.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:25 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:55:26.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:25 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:55:26.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:25 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:55:26.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 385242 29.17 MB/sec execute 581 sec latency 498.289 ms 2024-05-09T08:55:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:26 smithi160 ceph-mon[34070]: pgmap v1142: 97 pgs: 97 active+clean; 831 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-05-09T08:55:27.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:26 smithi133 ceph-mon[29610]: pgmap v1142: 97 pgs: 97 active+clean; 831 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-05-09T08:55:27.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:26 smithi188 ceph-mon[34645]: pgmap v1142: 97 pgs: 97 active+clean; 831 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-05-09T08:55:27.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 385737 29.16 MB/sec execute 582 sec latency 510.291 ms 2024-05-09T08:55:28.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 386311 29.17 MB/sec execute 583 sec latency 60.959 ms 2024-05-09T08:55:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:28 smithi160 ceph-mon[34070]: pgmap v1143: 97 pgs: 97 active+clean; 815 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:55:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:55:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:55:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:55:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:28 smithi133 ceph-mon[29610]: pgmap v1143: 97 pgs: 97 active+clean; 815 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:55:29.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:55:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:55:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:55:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:28 smithi188 ceph-mon[34645]: pgmap v1143: 97 pgs: 97 active+clean; 815 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-05-09T08:55:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:55:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:55:29.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:55:29.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 386930 29.19 MB/sec execute 584 sec latency 70.881 ms 2024-05-09T08:55:30.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 387413 29.18 MB/sec execute 585 sec latency 82.461 ms 2024-05-09T08:55:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:30 smithi160 ceph-mon[34070]: pgmap v1144: 97 pgs: 97 active+clean; 805 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 705 op/s 2024-05-09T08:55:31.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:30 smithi133 ceph-mon[29610]: pgmap v1144: 97 pgs: 97 active+clean; 805 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 705 op/s 2024-05-09T08:55:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:30 smithi188 ceph-mon[34645]: pgmap v1144: 97 pgs: 97 active+clean; 805 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 705 op/s 2024-05-09T08:55:31.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 387886 29.18 MB/sec execute 586 sec latency 115.368 ms 2024-05-09T08:55:32.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 388366 29.17 MB/sec execute 587 sec latency 79.049 ms 2024-05-09T08:55:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:32 smithi160 ceph-mon[34070]: pgmap v1145: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-05-09T08:55:33.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:32 smithi133 ceph-mon[29610]: pgmap v1145: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-05-09T08:55:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:32 smithi188 ceph-mon[34645]: pgmap v1145: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 706 op/s 2024-05-09T08:55:33.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 388858 29.15 MB/sec execute 588 sec latency 80.212 ms 2024-05-09T08:55:34.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 389593 29.17 MB/sec execute 589 sec latency 66.139 ms 2024-05-09T08:55:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:34 smithi160 ceph-mon[34070]: pgmap v1146: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-05-09T08:55:35.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:34 smithi133 ceph-mon[29610]: pgmap v1146: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-05-09T08:55:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:34 smithi188 ceph-mon[34645]: pgmap v1146: 97 pgs: 97 active+clean; 802 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-05-09T08:55:35.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 390309 29.18 MB/sec execute 590 sec latency 47.246 ms 2024-05-09T08:55:36.417 INFO:tasks.workunit.client.0.smithi133.stdout: 10 390922 29.21 MB/sec execute 591 sec latency 74.673 ms 2024-05-09T08:55:37.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:36 smithi160 ceph-mon[34070]: pgmap v1147: 97 pgs: 97 active+clean; 786 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:55:37.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:36 smithi133 ceph-mon[29610]: pgmap v1147: 97 pgs: 97 active+clean; 786 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:55:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:36 smithi188 ceph-mon[34645]: pgmap v1147: 97 pgs: 97 active+clean; 786 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 654 op/s 2024-05-09T08:55:37.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 391496 29.22 MB/sec execute 592 sec latency 47.792 ms 2024-05-09T08:55:38.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 391907 29.20 MB/sec execute 593 sec latency 118.143 ms 2024-05-09T08:55:39.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:38 smithi160 ceph-mon[34070]: pgmap v1148: 97 pgs: 97 active+clean; 791 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 548 op/s 2024-05-09T08:55:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:38 smithi133 ceph-mon[29610]: pgmap v1148: 97 pgs: 97 active+clean; 791 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 548 op/s 2024-05-09T08:55:39.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:38 smithi188 ceph-mon[34645]: pgmap v1148: 97 pgs: 97 active+clean; 791 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 548 op/s 2024-05-09T08:55:39.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 392384 29.19 MB/sec execute 594 sec latency 121.733 ms 2024-05-09T08:55:40.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 392875 29.18 MB/sec execute 595 sec latency 108.182 ms 2024-05-09T08:55:41.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:40 smithi160 ceph-mon[34070]: pgmap v1149: 97 pgs: 97 active+clean; 794 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-05-09T08:55:41.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:40 smithi133 ceph-mon[29610]: pgmap v1149: 97 pgs: 97 active+clean; 794 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-05-09T08:55:41.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:40 smithi188 ceph-mon[34645]: pgmap v1149: 97 pgs: 97 active+clean; 794 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-05-09T08:55:41.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 393441 29.18 MB/sec execute 596 sec latency 131.004 ms 2024-05-09T08:55:42.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 394127 29.19 MB/sec execute 597 sec latency 61.019 ms 2024-05-09T08:55:43.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:42 smithi160 ceph-mon[34070]: pgmap v1150: 97 pgs: 97 active+clean; 802 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-05-09T08:55:43.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:42 smithi133 ceph-mon[29610]: pgmap v1150: 97 pgs: 97 active+clean; 802 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-05-09T08:55:43.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:42 smithi188 ceph-mon[34645]: pgmap v1150: 97 pgs: 97 active+clean; 802 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 468 op/s 2024-05-09T08:55:43.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 394729 29.20 MB/sec execute 598 sec latency 66.710 ms 2024-05-09T08:55:44.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 395359 29.21 MB/sec execute 599 sec latency 77.622 ms 2024-05-09T08:55:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:44 smithi160 ceph-mon[34070]: pgmap v1151: 97 pgs: 97 active+clean; 803 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:55:45.176 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:44 smithi133 ceph-mon[29610]: pgmap v1151: 97 pgs: 97 active+clean; 803 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:55:45.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:44 smithi188 ceph-mon[34645]: pgmap v1151: 97 pgs: 97 active+clean; 803 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.3 MiB/s wr, 410 op/s 2024-05-09T08:55:45.418 INFO:tasks.workunit.client.0.smithi133.stdout: 10 cleanup 600 sec 2024-05-09T08:55:46.419 INFO:tasks.workunit.client.0.smithi133.stdout: 8 cleanup 601 sec 2024-05-09T08:55:47.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:46 smithi160 ceph-mon[34070]: pgmap v1152: 97 pgs: 97 active+clean; 811 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:55:47.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:46 smithi133 ceph-mon[29610]: pgmap v1152: 97 pgs: 97 active+clean; 811 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:55:47.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:46 smithi188 ceph-mon[34645]: pgmap v1152: 97 pgs: 97 active+clean; 811 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.4 MiB/s wr, 403 op/s 2024-05-09T08:55:47.419 INFO:tasks.workunit.client.0.smithi133.stdout: 1 cleanup 602 sec 2024-05-09T08:55:47.545 INFO:tasks.workunit.client.0.smithi133.stdout: 0 cleanup 602 sec 2024-05-09T08:55:47.545 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:55:47.545 INFO:tasks.workunit.client.0.smithi133.stdout: Operation Count AvgLat MaxLat 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: ---------------------------------------- 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: NTCreateX 561107 4.421 1232.584 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Close 412041 0.007 2.940 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Rename 23775 19.946 1111.130 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Unlink 113405 6.752 913.632 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Deltree 4 160.552 190.901 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Mkdir 2 0.022 0.024 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Qpathinfo 508924 3.195 1419.967 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Qfileinfo 88844 0.003 0.955 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Qfsinfo 93267 0.473 131.592 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Sfileinfo 45697 0.227 203.356 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: Find 196579 0.505 789.085 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: WriteX 277895 0.076 386.487 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: ReadX 880122 0.032 370.873 2024-05-09T08:55:47.546 INFO:tasks.workunit.client.0.smithi133.stdout: LockX 1830 5.101 284.930 2024-05-09T08:55:47.547 INFO:tasks.workunit.client.0.smithi133.stdout: UnlockX 1830 4.600 315.323 2024-05-09T08:55:47.547 INFO:tasks.workunit.client.0.smithi133.stdout: Flush 39282 10.531 1074.253 2024-05-09T08:55:47.547 INFO:tasks.workunit.client.0.smithi133.stdout: 2024-05-09T08:55:47.547 INFO:tasks.workunit.client.0.smithi133.stdout:Throughput 29.2127 MB/sec 10 clients 10 procs max_latency=1419.975 ms 2024-05-09T08:55:47.626 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T08:55:47.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T08:55:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:48 smithi160 ceph-mon[34070]: pgmap v1153: 97 pgs: 97 active+clean; 814 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-05-09T08:55:49.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:48 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:48 smithi133 ceph-mon[29610]: pgmap v1153: 97 pgs: 97 active+clean; 814 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-05-09T08:55:49.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:48 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:48 smithi188 ceph-mon[34645]: pgmap v1153: 97 pgs: 97 active+clean; 814 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.2 MiB/s wr, 371 op/s 2024-05-09T08:55:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:48 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:50.040 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-09T08:55:50.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T08:55:51.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:50 smithi160 ceph-mon[34070]: pgmap v1154: 97 pgs: 97 active+clean; 736 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2024-05-09T08:55:51.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:50 smithi133 ceph-mon[29610]: pgmap v1154: 97 pgs: 97 active+clean; 736 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2024-05-09T08:55:51.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:50 smithi188 ceph-mon[34645]: pgmap v1154: 97 pgs: 97 active+clean; 736 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 355 op/s 2024-05-09T08:55:51.394 DEBUG:teuthology.parallel:result is None 2024-05-09T08:55:51.394 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:55:51.497 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:55:51.497 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T08:55:51.508 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.032+0000 7fd1dc800640 1 -- 172.21.15.133:0/3220730159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4105660 msgr2=0x7fd1d410bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.032+0000 7fd1dc800640 1 --2- 172.21.15.133:0/3220730159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4105660 0x7fd1d410bef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009ca0 tx=0x7fd1d002a8b0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.034+0000 7fd1dc800640 1 -- 172.21.15.133:0/3220730159 shutdown_connections 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.034+0000 7fd1dc800640 1 --2- 172.21.15.133:0/3220730159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4105660 0x7fd1d410bef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.034+0000 7fd1dc800640 1 --2- 172.21.15.133:0/3220730159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4104ca0 0x7fd1d4105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.034+0000 7fd1dc800640 1 --2- 172.21.15.133:0/3220730159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4103ab0 0x7fd1d4103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.034+0000 7fd1dc800640 1 -- 172.21.15.133:0/3220730159 >> 172.21.15.133:0/3220730159 conn(0x7fd1d40ff020 msgr2=0x7fd1d4101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:52.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.037+0000 7fd1dc800640 1 -- 172.21.15.133:0/3220730159 shutdown_connections 2024-05-09T08:55:52.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.037+0000 7fd1dc800640 1 -- 172.21.15.133:0/3220730159 wait complete. 2024-05-09T08:55:52.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.040+0000 7fd1dc800640 1 Processor -- start 2024-05-09T08:55:52.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.041+0000 7fd1dc800640 1 -- start start 2024-05-09T08:55:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 0x7fd1d419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 0x7fd1d419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 0x7fd1d41a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d410f760 con 0x7fd1d4103ab0 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d410f5e0 con 0x7fd1d4104ca0 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1dc800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d410f8e0 con 0x7fd1d4105660 2024-05-09T08:55:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.044+0000 7fd1d9400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 0x7fd1d419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1d9400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 0x7fd1d419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39616/0 (socket says 172.21.15.133:39616) 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1d9400640 1 -- 172.21.15.133:0/488852384 learned_addr learned my addr 172.21.15.133:0/488852384 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 0x7fd1d41a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1d9e00640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 0x7fd1d419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 -- 172.21.15.133:0/488852384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 msgr2=0x7fd1d419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 0x7fd1d419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 -- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 msgr2=0x7fd1d419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 0x7fd1d419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.045+0000 7fd1da800640 1 -- 172.21.15.133:0/488852384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d41a5d60 con 0x7fd1d4105660 2024-05-09T08:55:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.046+0000 7fd1d9e00640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 0x7fd1d419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:55:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.046+0000 7fd1da800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 0x7fd1d41a56f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0030860 tx=0x7fd1d0030cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.047+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0003d80 con 0x7fd1d4105660 2024-05-09T08:55:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.047+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0040070 con 0x7fd1d4105660 2024-05-09T08:55:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.047+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0004720 con 0x7fd1d4105660 2024-05-09T08:55:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.048+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d41a5ff0 con 0x7fd1d4105660 2024-05-09T08:55:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.048+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41a6460 con 0x7fd1d4105660 2024-05-09T08:55:52.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.050+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d41a6780 con 0x7fd1d4105660 2024-05-09T08:55:52.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.057+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd1d0031880 con 0x7fd1d4105660 2024-05-09T08:55:52.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.059+0000 7fd1cea00640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 0x7fd1bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.061+0000 7fd1d9e00640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 0x7fd1bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.061+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1d00effd0 con 0x7fd1d4105660 2024-05-09T08:55:52.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.064+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1d00bcf00 con 0x7fd1d4105660 2024-05-09T08:55:52.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.064+0000 7fd1d9e00640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 0x7fd1bc0a5150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80058c0 tx=0x7fd1c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:52.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.494+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1d41a6b10 con 0x7fd1d4105660 2024-05-09T08:55:52.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.496+0000 7fd1cea00640 1 -- 172.21.15.133:0/488852384 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7fd1d41a6b10 con 0x7fd1d4105660 2024-05-09T08:55:52.498 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:55:52.499 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 msgr2=0x7fd1bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 0x7fd1bc0a5150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80058c0 tx=0x7fd1c8005850 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 msgr2=0x7fd1d41a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 0x7fd1d41a56f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0030860 tx=0x7fd1d0030cd0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 shutdown_connections 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd1bc0a2c90 0x7fd1bc0a5150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d4105660 0x7fd1d41a56f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1d4104ca0 0x7fd1d419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 --2- 172.21.15.133:0/488852384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1d4103ab0 0x7fd1d419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.501+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 >> 172.21.15.133:0/488852384 conn(0x7fd1d40ff020 msgr2=0x7fd1d4077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:52.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.503+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 shutdown_connections 2024-05-09T08:55:52.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.503+0000 7fd1dc800640 1 -- 172.21.15.133:0/488852384 wait complete. 2024-05-09T08:55:52.525 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:55:52.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.759+0000 7fd72c800640 1 -- 172.21.15.133:0/3530946388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 msgr2=0x7fd7241144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.759+0000 7fd72c800640 1 --2- 172.21.15.133:0/3530946388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd7241144c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd720009c10 tx=0x7fd72002a6e0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 -- 172.21.15.133:0/3530946388 shutdown_connections 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 --2- 172.21.15.133:0/3530946388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd7241144c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 --2- 172.21.15.133:0/3530946388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd724111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 --2- 172.21.15.133:0/3530946388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd72410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 -- 172.21.15.133:0/3530946388 >> 172.21.15.133:0/3530946388 conn(0x7fd72406e6c0 msgr2=0x7fd724070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:52.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 -- 172.21.15.133:0/3530946388 shutdown_connections 2024-05-09T08:55:52.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.760+0000 7fd72c800640 1 -- 172.21.15.133:0/3530946388 wait complete. 2024-05-09T08:55:52.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.761+0000 7fd72c800640 1 Processor -- start 2024-05-09T08:55:52.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 -- start start 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd7241a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd724119140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd72410c8f0 con 0x7fd72410b1a0 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd72410c770 con 0x7fd72410b620 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd72c800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd72410ca70 con 0x7fd724108830 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.762+0000 7fd729e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:51592/0 (socket says 172.21.15.133:51592) 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729e00640 1 -- 172.21.15.133:0/3965264706 learned_addr learned my addr 172.21.15.133:0/3965264706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729400640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd7241a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729e00640 1 -- 172.21.15.133:0/3965264706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 msgr2=0x7fd724119140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd72a800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd724119140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729e00640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd724119140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.763+0000 7fd729e00640 1 -- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 msgr2=0x7fd7241a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.765+0000 7fd729e00640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd7241a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.765+0000 7fd729e00640 1 -- 172.21.15.133:0/3965264706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7241197b0 con 0x7fd724108830 2024-05-09T08:55:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.765+0000 7fd72a800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd724119140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:55:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.765+0000 7fd729400640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd7241a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:55:52.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.766+0000 7fd729e00640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:52.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.766+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714017070 con 0x7fd724108830 2024-05-09T08:55:52.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.766+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd724119aa0 con 0x7fd724108830 2024-05-09T08:55:52.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.766+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd714009c30 con 0x7fd724108830 2024-05-09T08:55:52.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.767+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f6b0 con 0x7fd724108830 2024-05-09T08:55:52.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.767+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd724119f30 con 0x7fd724108830 2024-05-09T08:55:52.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.768+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7241109a0 con 0x7fd724108830 2024-05-09T08:55:52.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.770+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd714010460 con 0x7fd724108830 2024-05-09T08:55:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.773+0000 7fd71ea00640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 0x7fd70c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:52.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.773+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd7140cf060 con 0x7fd724108830 2024-05-09T08:55:52.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.773+0000 7fd729400640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 0x7fd70c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.777+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd71409bf90 con 0x7fd724108830 2024-05-09T08:55:52.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:52.778+0000 7fd729400640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 0x7fd70c0a5150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd7241a7e50 tx=0x7fd718005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:53.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.051+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd724062430 con 0x7fd724108830 2024-05-09T08:55:53.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.051+0000 7fd71ea00640 1 -- 172.21.15.133:0/3965264706 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd714084e20 con 0x7fd724108830 2024-05-09T08:55:53.051 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:55:53.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 msgr2=0x7fd70c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 0x7fd70c0a5150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd7241a7e50 tx=0x7fd718005f90 comp rx=0 tx=0).stop 2024-05-09T08:55:53.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 msgr2=0x7fd7241a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400de10 comp rx=0 tx=0).stop 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 shutdown_connections 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fd70c0a2c90 0x7fd70c0a5150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72410b620 0x7fd724119140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd72410b1a0 0x7fd7241a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 --2- 172.21.15.133:0/3965264706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd724108830 0x7fd7241a6940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 >> 172.21.15.133:0/3965264706 conn(0x7fd72406e6c0 msgr2=0x7fd7240708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.056+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 shutdown_connections 2024-05-09T08:55:53.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.057+0000 7fd72c800640 1 -- 172.21.15.133:0/3965264706 wait complete. 2024-05-09T08:55:53.077 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:52 smithi133 ceph-mon[29610]: pgmap v1155: 97 pgs: 97 active+clean; 657 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 381 op/s 2024-05-09T08:55:53.078 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:52 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:53.078 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:52 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/488852384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:55:53.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:55:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:52 smithi160 ceph-mon[34070]: pgmap v1155: 97 pgs: 97 active+clean; 657 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 381 op/s 2024-05-09T08:55:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:52 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:53.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:52 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/488852384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:55:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:52 smithi188 ceph-mon[34645]: pgmap v1155: 97 pgs: 97 active+clean; 657 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 381 op/s 2024-05-09T08:55:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:52 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:53.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:52 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/488852384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.293+0000 7f53ea000640 1 -- 172.21.15.133:0/4261539407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e41021c0 msgr2=0x7f53e41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.293+0000 7f53ea000640 1 --2- 172.21.15.133:0/4261539407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e41021c0 0x7f53e41045b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009900 tx=0x7f53d802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 -- 172.21.15.133:0/4261539407 shutdown_connections 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 --2- 172.21.15.133:0/4261539407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e4107420 0x7f53e4109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 --2- 172.21.15.133:0/4261539407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4104af0 0x7f53e4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 --2- 172.21.15.133:0/4261539407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e41021c0 0x7f53e41045b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 -- 172.21.15.133:0/4261539407 >> 172.21.15.133:0/4261539407 conn(0x7f53e40fbbd0 msgr2=0x7f53e40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 -- 172.21.15.133:0/4261539407 shutdown_connections 2024-05-09T08:55:53.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.294+0000 7f53ea000640 1 -- 172.21.15.133:0/4261539407 wait complete. 2024-05-09T08:55:53.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 Processor -- start 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 -- start start 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e41021c0 0x7f53e4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4107420 0x7f53e419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410d410 con 0x7f53e41021c0 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410d290 con 0x7f53e4104af0 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53ea000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53e410d590 con 0x7f53e4107420 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53e3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53e8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e41021c0 0x7f53e4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53e3e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39658/0 (socket says 172.21.15.133:39658) 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53e3e00640 1 -- 172.21.15.133:0/3544732190 learned_addr learned my addr 172.21.15.133:0/3544732190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.295+0000 7f53e9600640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4107420 0x7f53e419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 -- 172.21.15.133:0/3544732190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4107420 msgr2=0x7f53e419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4107420 0x7f53e419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 -- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e41021c0 msgr2=0x7f53e4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e41021c0 0x7f53e4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 -- 172.21.15.133:0/3544732190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e419f6b0 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e3e00640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f53d000e9a0 tx=0x7f53d000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000cd80 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e419f9a0 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53d00035f0 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0010670 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.296+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e419ff40 con 0x7f53e4104af0 2024-05-09T08:55:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.297+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e40fd6c0 con 0x7f53e4104af0 2024-05-09T08:55:53.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.300+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f53d00037a0 con 0x7f53e4104af0 2024-05-09T08:55:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.301+0000 7f53e1600640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 0x7f53cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.301+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f53d00cf330 con 0x7f53e4104af0 2024-05-09T08:55:53.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.301+0000 7f53e8c00640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 0x7f53cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.302+0000 7f53e8c00640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 0x7f53cc0a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53d8031440 tx=0x7f53d80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:53.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.303+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f53d009c260 con 0x7f53e4104af0 2024-05-09T08:55:53.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.564+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f53e402d410 con 0x7f53e4104af0 2024-05-09T08:55:53.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.565+0000 7f53e1600640 1 -- 172.21.15.133:0/3544732190 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f53d009bb60 con 0x7f53e4104af0 2024-05-09T08:55:53.566 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:55:53.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 msgr2=0x7f53cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 0x7f53cc0a50d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f53d8031440 tx=0x7f53d80315b0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 msgr2=0x7f53e4198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f53d000e9a0 tx=0x7f53d000ee60 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 shutdown_connections 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f53cc0a2c10 0x7f53cc0a50d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53e4107420 0x7f53e419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53e4104af0 0x7f53e4198040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 --2- 172.21.15.133:0/3544732190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e41021c0 0x7f53e4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 >> 172.21.15.133:0/3544732190 conn(0x7f53e40fbbd0 msgr2=0x7f53e4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 shutdown_connections 2024-05-09T08:55:53.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.568+0000 7f53ea000640 1 -- 172.21.15.133:0/3544732190 wait complete. 2024-05-09T08:55:53.587 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:55:53.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 -- 172.21.15.133:0/1378285154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381029b0 msgr2=0x7fa738102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 --2- 172.21.15.133:0/1378285154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381029b0 0x7fa738102e30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa720009900 tx=0x7fa72002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 -- 172.21.15.133:0/1378285154 shutdown_connections 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 --2- 172.21.15.133:0/1378285154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 0x7fa738109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 --2- 172.21.15.133:0/1378285154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381029b0 0x7fa738102e30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 --2- 172.21.15.133:0/1378285154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381017c0 0x7fa738101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 -- 172.21.15.133:0/1378285154 >> 172.21.15.133:0/1378285154 conn(0x7fa7380fcd30 msgr2=0x7fa7380ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 -- 172.21.15.133:0/1378285154 shutdown_connections 2024-05-09T08:55:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.767+0000 7fa73ca00640 1 -- 172.21.15.133:0/1378285154 wait complete. 2024-05-09T08:55:53.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 Processor -- start 2024-05-09T08:55:53.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 -- start start 2024-05-09T08:55:53.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381029b0 0x7fa738197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 0x7fa73819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d420 con 0x7fa738103370 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d2a0 con 0x7fa7381017c0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.768+0000 7fa73ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d5a0 con 0x7fa7381029b0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:39666/0 (socket says 172.21.15.133:39666) 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 -- 172.21.15.133:0/2906813146 learned_addr learned my addr 172.21.15.133:0/2906813146 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa736a00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381029b0 0x7fa738197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737e00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 0x7fa73819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 -- 172.21.15.133:0/2906813146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381029b0 msgr2=0x7fa738197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381029b0 0x7fa738197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 -- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 msgr2=0x7fa73819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 0x7fa73819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 -- 172.21.15.133:0/2906813146 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73819f6b0 con 0x7fa7381017c0 2024-05-09T08:55:53.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa737400640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9c0 tx=0x7fa72800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800cd80 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7280035f0 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.769+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73819f9a0 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.770+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728010690 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.770+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73819fe70 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.771+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa738108ae0 con 0x7fa7381017c0 2024-05-09T08:55:53.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.772+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa7280037a0 con 0x7fa7381017c0 2024-05-09T08:55:53.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.774+0000 7fa72fe00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 0x7fa71c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:53.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.774+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa7280cf520 con 0x7fa7381017c0 2024-05-09T08:55:53.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.774+0000 7fa736a00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 0x7fa71c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:53.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.775+0000 7fa736a00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 0x7fa71c0a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa720006010 tx=0x7fa720000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:53.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.778+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa72809c450 con 0x7fa7381017c0 2024-05-09T08:55:53.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:53 smithi133 ceph-mon[29610]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:53.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:53 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3965264706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:55:53.841 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:53 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3544732190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:55:53.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.974+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa738062460 con 0x7fa7381017c0 2024-05-09T08:55:53.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.975+0000 7fa72fe00640 1 -- 172.21.15.133:0/2906813146 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fa72809bd50 con 0x7fa7381017c0 2024-05-09T08:55:53.975 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 msgr2=0x7fa71c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 0x7fa71c0a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa720006010 tx=0x7fa720000bc0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 msgr2=0x7fa738197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9c0 tx=0x7fa72800ee80 comp rx=0 tx=0).stop 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 shutdown_connections 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa71c0a2b90 0x7fa71c0a5050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa738103370 0x7fa73819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7381029b0 0x7fa738197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 --2- 172.21.15.133:0/2906813146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7381017c0 0x7fa738197a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:53.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 >> 172.21.15.133:0/2906813146 conn(0x7fa7380fcd30 msgr2=0x7fa738105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:53.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 shutdown_connections 2024-05-09T08:55:53.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:53.977+0000 7fa73ca00640 1 -- 172.21.15.133:0/2906813146 wait complete. 2024-05-09T08:55:53.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T08:55:54.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:53 smithi160 ceph-mon[34070]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:54.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:53 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3965264706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:55:54.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:53 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3544732190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:55:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:53 smithi188 ceph-mon[34645]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T08:55:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:53 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3965264706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:55:54.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:53 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3544732190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 -- 172.21.15.133:0/1294989149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 msgr2=0x7f1370074670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 --2- 172.21.15.133:0/1294989149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370074670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1358009900 tx=0x7f135802a490 comp rx=0 tx=0).stop 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 -- 172.21.15.133:0/1294989149 shutdown_connections 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 --2- 172.21.15.133:0/1294989149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1370109730 0x7f137010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 --2- 172.21.15.133:0/1294989149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370074670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 --2- 172.21.15.133:0/1294989149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13700757f0 0x7f1370073c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 -- 172.21.15.133:0/1294989149 >> 172.21.15.133:0/1294989149 conn(0x7f13700fce10 msgr2=0x7f13700ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 -- 172.21.15.133:0/1294989149 shutdown_connections 2024-05-09T08:55:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.186+0000 7f1375200640 1 -- 172.21.15.133:0/1294989149 wait complete. 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.187+0000 7f1375200640 1 Processor -- start 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.187+0000 7f1375200640 1 -- start start 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13700757f0 0x7f137019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1370109730 0x7f13701a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f710 con 0x7f1370074210 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f590 con 0x7f1370109730 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1375200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f137010f890 con 0x7f13700757f0 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f1365400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13700757f0 0x7f137019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51566/0 (socket says 172.21.15.133:51566) 2024-05-09T08:55:54.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 -- 172.21.15.133:0/3560541311 learned_addr learned my addr 172.21.15.133:0/3560541311 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136e800640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1370109730 0x7f13701a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 -- 172.21.15.133:0/3560541311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13700757f0 msgr2=0x7f137019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13700757f0 0x7f137019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 -- 172.21.15.133:0/3560541311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1370109730 msgr2=0x7f13701a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1370109730 0x7f13701a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 -- 172.21.15.133:0/3560541311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13701a19d0 con 0x7f1370074210 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.188+0000 7f136de00640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.189+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000cd80 con 0x7f1370074210 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.189+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13701a1cc0 con 0x7f1370074210 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.189+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1360010910 con 0x7f1370074210 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.189+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600188f0 con 0x7f1370074210 2024-05-09T08:55:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.189+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a21b0 con 0x7f1370074210 2024-05-09T08:55:54.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.190+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13701084f0 con 0x7f1370074210 2024-05-09T08:55:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.193+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1360010ac0 con 0x7f1370074210 2024-05-09T08:55:54.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.194+0000 7f1367400640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 0x7f13540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.194+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f13600cf420 con 0x7f1370074210 2024-05-09T08:55:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.194+0000 7f1365400640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 0x7f13540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.195+0000 7f1365400640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 0x7f13540a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13580314e0 tx=0x7f1358000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:54.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.199+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f136009bd00 con 0x7f1370074210 2024-05-09T08:55:54.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.441+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f13700624d0 con 0x7f1370074210 2024-05-09T08:55:54.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.475+0000 7f1367400640 1 -- 172.21.15.133:0/3560541311 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740376 (secure 0 0 0) 0x7f13600a0000 con 0x7f1370074210 2024-05-09T08:55:54.493 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:55:54.493 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:55:54.493 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 20, 2024-05-09T08:55:54.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.493 INFO:teuthology.orchestra.run.smithi133.stdout: "active_gid": 14152, 2024-05-09T08:55:54.494 INFO:teuthology.orchestra.run.smithi133.stdout: "active_name": "x", 2024-05-09T08:55:54.494 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addrs": { 2024-05-09T08:55:54.494 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-05-09T08:55:54.494 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6800", 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2058981411 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-05-09T08:55:54.495 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6801", 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2058981411 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addr": "172.21.15.133:6801/2058981411", 2024-05-09T08:55:54.496 INFO:teuthology.orchestra.run.smithi133.stdout: "active_change": "2024-05-09T08:19:56.515822+0000", 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "available": true, 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "standbys": [ 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24109, 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "z", 2024-05-09T08:55:54.497 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540701547738038271, 2024-05-09T08:55:54.498 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-05-09T08:55:54.498 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-05-09T08:55:54.498 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.498 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.499 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.500 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T08:55:54.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.502 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.503 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.505 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-05-09T08:55:54.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T08:55:54.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.520 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.525 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.527 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T08:55:54.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:54.533 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T08:55:54.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.538 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.545 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-05-09T08:55:54.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-05-09T08:55:54.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-05-09T08:55:54.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.549 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.550 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.553 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.554 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-05-09T08:55:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T08:55:54.557 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.559 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T08:55:54.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-05-09T08:55:54.563 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-05-09T08:55:54.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-05-09T08:55:54.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-05-09T08:55:54.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-05-09T08:55:54.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.570 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.572 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.574 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-05-09T08:55:54.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.578 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.582 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.586 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T08:55:54.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.596 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.597 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T08:55:54.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.600 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T08:55:54.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T08:55:54.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.605 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.607 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T08:55:54.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:54.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.615 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.617 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T08:55:54.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.619 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.622 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.632 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.635 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T08:55:54.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T08:55:54.637 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T08:55:54.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.639 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-05-09T08:55:54.640 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-05-09T08:55:54.641 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.643 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.644 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.645 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.646 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.648 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.649 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.654 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.655 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.657 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.658 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.659 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.661 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.663 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-05-09T08:55:54.666 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.668 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T08:55:54.672 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.673 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.676 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.677 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.678 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-05-09T08:55:54.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-05-09T08:55:54.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-05-09T08:55:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.686 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.689 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-05-09T08:55:54.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-05-09T08:55:54.691 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.693 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.696 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.697 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.701 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.702 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.704 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.705 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-05-09T08:55:54.707 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-05-09T08:55:54.708 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.710 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.712 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-05-09T08:55:54.714 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.716 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.717 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.718 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.719 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.721 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.723 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.724 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.726 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.730 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.731 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.732 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.734 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.735 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.736 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.737 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.738 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.739 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.740 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.741 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.742 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.744 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.745 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.746 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-05-09T08:55:54.748 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.749 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.750 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.753 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.755 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.758 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.759 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.764 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.765 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.766 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.767 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.769 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.770 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.771 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.772 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-05-09T08:55:54.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.774 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-05-09T08:55:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.776 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.777 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.778 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.780 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.781 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.782 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.783 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.784 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.785 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.786 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.787 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-05-09T08:55:54.788 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.789 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.790 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.791 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.792 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.793 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.794 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.795 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.796 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.799 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.800 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.801 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.802 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.803 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.804 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.805 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.806 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.807 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.808 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.809 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.810 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.811 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.812 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.813 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.814 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.815 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.816 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.817 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.818 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.819 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.820 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.821 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.822 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.823 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.824 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.825 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.826 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.827 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.828 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.829 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.830 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-05-09T08:55:54.831 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-05-09T08:55:54.832 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.833 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.834 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.835 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.836 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.838 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.839 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.840 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.841 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.842 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.843 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.844 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.845 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.846 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.847 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.849 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.850 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.851 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.852 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.853 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-05-09T08:55:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-05-09T08:55:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.870 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-05-09T08:55:54.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-05-09T08:55:54.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.874 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.875 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.876 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.877 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.880 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.882 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.883 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.884 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.885 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.886 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.887 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.888 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.890 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.891 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-05-09T08:55:54.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.893 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.895 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.896 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.898 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.899 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.901 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.902 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.903 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.904 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T08:55:54.905 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.906 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24110, 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "y", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540701547738038271, 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.908 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.909 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.911 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.912 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.913 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-05-09T08:55:54.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.915 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.917 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-05-09T08:55:54.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-05-09T08:55:54.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.922 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T08:55:54.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.924 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.925 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-05-09T08:55:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.928 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.929 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.930 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.931 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.932 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.933 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-05-09T08:55:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.939 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T08:55:54.941 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T08:55:54.942 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.943 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.945 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T08:55:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T08:55:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-05-09T08:55:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-05-09T08:55:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-05-09T08:55:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-05-09T08:55:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-05-09T08:55:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.960 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T08:55:54.961 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.963 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.964 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T08:55:54.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.967 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-05-09T08:55:54.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.970 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T08:55:54.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.973 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.974 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.975 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T08:55:54.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.977 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.978 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.980 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.981 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.983 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.984 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.985 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.986 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.987 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-05-09T08:55:54.988 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.990 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T08:55:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-05-09T08:55:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.004 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.005 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.006 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T08:55:55.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.010 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.012 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.014 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.015 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T08:55:55.017 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T08:55:55.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.019 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.021 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.023 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.025 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T08:55:55.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.031 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.034 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.035 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.036 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.038 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.040 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.041 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.042 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.043 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.044 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.046 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.048 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.049 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.050 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.052 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.053 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.054 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.055 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.057 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.058 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.060 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.061 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.062 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.063 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.064 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.065 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.066 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.068 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.071 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-05-09T08:55:55.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.073 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.076 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.077 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.078 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.079 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.080 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-05-09T08:55:55.081 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.083 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.084 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.086 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.088 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.089 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.090 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.091 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.092 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.093 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-05-09T08:55:55.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.096 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.098 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.099 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.100 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.103 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.104 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.107 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.108 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.109 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.111 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.112 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.113 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.114 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.118 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.119 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.120 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.121 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-05-09T08:55:55.123 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.125 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.126 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.127 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.129 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.130 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.131 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.132 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.134 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.135 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.136 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.137 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.138 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.139 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.140 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.141 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.142 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.143 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.144 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.145 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.147 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-05-09T08:55:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.149 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.150 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.151 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.152 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.153 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.154 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.155 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.157 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.158 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.159 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.160 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.161 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.167 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.172 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.175 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-05-09T08:55:55.176 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.177 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.181 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.184 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.186 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-05-09T08:55:55.187 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.188 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.189 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:55.190 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.191 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.192 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.193 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.194 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.195 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:55.196 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:55.199 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.200 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.201 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.202 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.203 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.204 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.205 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.206 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.207 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T08:55:55.208 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.209 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.210 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.211 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.212 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.213 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.214 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.215 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.216 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.218 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-05-09T08:55:55.219 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.220 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.221 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.222 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.223 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.225 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.226 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.227 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.228 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.229 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.230 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.231 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.232 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.233 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.234 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.235 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-05-09T08:55:55.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.238 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.240 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.241 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.242 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.243 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.244 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.245 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.246 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.247 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.248 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.249 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.250 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.251 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.252 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.253 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.254 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.255 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.255 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.255 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.255 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.256 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.257 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.258 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.260 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.261 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.263 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.264 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.265 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.266 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-05-09T08:55:55.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.268 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.269 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.272 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.273 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-05-09T08:55:55.274 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.275 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.276 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.277 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.278 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.279 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.280 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.281 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.282 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-05-09T08:55:55.283 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-05-09T08:55:55.284 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.285 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.288 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.289 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.291 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.292 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.293 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.295 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.298 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.299 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.300 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.302 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-05-09T08:55:55.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-05-09T08:55:55.305 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.306 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.307 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.309 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.310 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.311 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.312 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.313 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.314 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.315 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.316 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "modules": [ 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard", 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "iostat", 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "nfs", 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "restful" 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.318 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.319 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.323 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2024-05-09T08:55:55.324 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2024-05-09T08:55:55.326 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2024-05-09T08:55:55.327 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2024-05-09T08:55:55.328 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2024-05-09T08:55:55.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2024-05-09T08:55:55.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.331 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2024-05-09T08:55:55.332 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.333 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:55.334 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.335 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:55.336 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.337 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T08:55:55.339 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2024-05-09T08:55:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.341 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.342 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2024-05-09T08:55:55.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.344 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.345 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.346 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.347 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.348 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.350 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.351 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "read", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap-read" 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2024-05-09T08:55:55.352 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.353 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2024-05-09T08:55:55.354 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.355 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.356 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.362 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm_log_destination": { 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm_log_destination", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "file", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "file,syslog", 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "syslog" 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.363 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.364 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.365 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2024-05-09T08:55:55.366 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.367 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T08:55:55.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.369 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.370 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.371 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.372 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.373 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.374 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.375 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.376 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2024-05-09T08:55:55.377 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_nvmeof": { 2024-05-09T08:55:55.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_nvmeof", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Nvme-of container image", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.379 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_samba": { 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_samba", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.383 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Samba/SMB container image", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T08:55:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.385 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T08:55:55.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.388 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.389 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.391 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.392 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.393 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "grafana_dashboards_path": { 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "grafana_dashboards_path", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.394 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.395 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.396 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "hw_monitoring": { 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hw_monitoring", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T08:55:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "inventory_list_all": { 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "inventory_list_all", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.399 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.400 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.402 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.403 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T08:55:55.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T08:55:55.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.406 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.407 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2024-05-09T08:55:55.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.409 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.410 INFO:teuthology.orchestra.run.smithi133.stdout: "oob_default_addr": { 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "oob_default_addr", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "169.254.1.1", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.411 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2024-05-09T08:55:55.412 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.413 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.414 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.415 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.416 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.417 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.420 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2024-05-09T08:55:55.422 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.424 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.425 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.426 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.427 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.428 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.429 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.431 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.432 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T08:55:55.435 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.436 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.437 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.439 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.440 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.441 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T08:55:55.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T08:55:55.443 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.445 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.446 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.450 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.452 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T08:55:55.453 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T08:55:55.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.456 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.458 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.459 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T08:55:55.460 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.461 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.463 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.464 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.465 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.468 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.471 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T08:55:55.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2024-05-09T08:55:55.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.481 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2024-05-09T08:55:55.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.490 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.491 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.493 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.495 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.496 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.498 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.502 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.503 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2024-05-09T08:55:55.506 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2024-05-09T08:55:55.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.509 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.510 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T08:55:55.510 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2024-05-09T08:55:55.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.515 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T08:55:55.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.517 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.518 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.519 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.520 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.522 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.523 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.525 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2024-05-09T08:55:55.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2024-05-09T08:55:55.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.530 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.532 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.537 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2024-05-09T08:55:55.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.540 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.543 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.544 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.545 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.546 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.548 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.550 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.553 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.554 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.557 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.559 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.562 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.563 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2024-05-09T08:55:55.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2024-05-09T08:55:55.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T08:55:55.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.568 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.570 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.571 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.573 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2024-05-09T08:55:55.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.575 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.577 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2024-05-09T08:55:55.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.580 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.581 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.582 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.584 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.585 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.589 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.594 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.596 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.599 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2024-05-09T08:55:55.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.601 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.608 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.610 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.611 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.613 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.615 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2024-05-09T08:55:55.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.619 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.620 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.621 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.624 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.626 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.627 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2024-05-09T08:55:55.628 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.630 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.632 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.633 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.635 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.637 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.638 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2024-05-09T08:55:55.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2024-05-09T08:55:55.640 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-05-09T08:55:55.641 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.642 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.643 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.644 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.645 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.647 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2024-05-09T08:55:55.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.649 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.652 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.654 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.655 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.656 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.658 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.659 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.661 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.662 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.664 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.665 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.666 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.669 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.671 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.674 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2024-05-09T08:55:55.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.678 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.680 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.681 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.682 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.685 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.686 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.688 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.689 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.691 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.693 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.694 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.695 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.696 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.697 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.698 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.701 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.702 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.703 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.704 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.705 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.707 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.708 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2024-05-09T08:55:55.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.713 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.718 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.719 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.721 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.722 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.723 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.724 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.726 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.726 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.726 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.726 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.726 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.729 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.730 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.731 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2024-05-09T08:55:55.732 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.734 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.737 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.738 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.739 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.741 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.742 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.744 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.745 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.746 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.747 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2024-05-09T08:55:55.748 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "periodic_async_work": { 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "periodic_async_work", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.749 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Periodically check for async work", 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.750 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_no_wait": { 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.752 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2024-05-09T08:55:55.753 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.755 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.756 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2024-05-09T08:55:55.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.758 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.760 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2024-05-09T08:55:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: "services": { 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard": "https://172.21.15.133:8443/" 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "always_on_modules": { 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "octopus": [ 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "pacific": [ 2024-05-09T08:55:55.764 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "quincy": [ 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-05-09T08:55:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "reef": [ 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-05-09T08:55:55.766 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "squid": [ 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2024-05-09T08:55:55.767 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "last_failure_osd_epoch": 3, 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "active_clients": [ 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3360446484 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "libcephsqlite", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 274371222 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-05-09T08:55:55.769 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 648550069 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 454568591 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:55.770 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-05-09T08:55:55.771 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:54 smithi188 ceph-mon[34645]: pgmap v1156: 97 pgs: 97 active+clean; 643 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 341 op/s 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:54 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2906813146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:54 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3560541311' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:54 smithi160 ceph-mon[34070]: pgmap v1156: 97 pgs: 97 active+clean; 643 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 341 op/s 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:54 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2906813146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:55:55.772 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:54 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3560541311' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 msgr2=0x7f13540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 0x7f13540a50d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f13580314e0 tx=0x7f1358000bc0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 msgr2=0x7f1370199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9f0 tx=0x7f136000eeb0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 shutdown_connections 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f13540a2c10 0x7f13540a50d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1370109730 0x7f13701a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13700757f0 0x7f137019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 --2- 172.21.15.133:0/3560541311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1370074210 0x7f1370199dc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.490+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 >> 172.21.15.133:0/3560541311 conn(0x7f13700fce10 msgr2=0x7f13701056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.491+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 shutdown_connections 2024-05-09T08:55:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:55:54.491+0000 7f1375200640 1 -- 172.21.15.133:0/3560541311 wait complete. 2024-05-09T08:55:55.778 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T08:55:55.779 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T08:55:55.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:54 smithi133 ceph-mon[29610]: pgmap v1156: 97 pgs: 97 active+clean; 643 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 341 op/s 2024-05-09T08:55:55.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:54 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2906813146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:55:55.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:54 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3560541311' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T08:55:56.222 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:55:56.918 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:56 smithi133 ceph-mon[29610]: pgmap v1157: 97 pgs: 97 active+clean; 616 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-05-09T08:55:57.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:56 smithi160 ceph-mon[34070]: pgmap v1157: 97 pgs: 97 active+clean; 616 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-05-09T08:55:57.192 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:56 smithi188 ceph-mon[34645]: pgmap v1157: 97 pgs: 97 active+clean; 616 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.7 MiB/s wr, 431 op/s 2024-05-09T08:55:59.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:55:58 smithi160 ceph-mon[34070]: pgmap v1158: 97 pgs: 97 active+clean; 609 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 402 op/s 2024-05-09T08:55:59.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:55:58 smithi133 ceph-mon[29610]: pgmap v1158: 97 pgs: 97 active+clean; 609 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 402 op/s 2024-05-09T08:55:59.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:55:58 smithi188 ceph-mon[34645]: pgmap v1158: 97 pgs: 97 active+clean; 609 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 402 op/s 2024-05-09T08:55:59.343 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:55:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:55:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 171074, 2024-05-09T08:55:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 149360, 2024-05-09T08:55:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 43325138, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 156883632, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 22.224368265, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 9.295890990, 2024-05-09T08:55:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 12.955469337, 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 2.371966136, 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 149360, 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 14.872566940, 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000099575 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 251614, 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.069205859, 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000275 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:55:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 315226, 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 479627, 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 115503090, 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 515898580, 2024-05-09T08:55:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 15, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 83.195756333, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 57.367417809, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 30.786886850, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.705842103, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 479626, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 52.078419809, 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000108581 2024-05-09T08:55:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 657751, 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.260195628, 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000395 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 108031152, 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 388902592 2024-05-09T08:55:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 90364, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 99152, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 26509314, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 104117077, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 15.129109743, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 8.060420575, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 6.771401884, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.294958189, 2024-05-09T08:55:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 99149, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 10.601804424, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000106928 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 136945, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.038264213, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000279 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 887632, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 283872 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-05-09T08:55:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 95126, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 140.463491182, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001476604 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-05-09T08:55:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 53977, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 9.931384130, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000183992 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 210086, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 193045, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 193045, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 655.536849168, 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003395772 2024-05-09T08:55:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 17041, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 52, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 3, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 3, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 443, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 215, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 107, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 121, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 134, 2024-05-09T08:55:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 82, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 1, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 1, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 4, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 346939, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 183680, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 17041, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 16, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 4518, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 5, 2024-05-09T08:55:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 3, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 5, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 129, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 171, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 35874872, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:55:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 2, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 3970, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 126369, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 52553, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 27075, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 11231, 2024-05-09T08:55:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 137317 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 5, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 33, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 22, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 28, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 14, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 34182, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 34182, 2024-05-09T08:55:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 5, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 6, 2024-05-09T08:55:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 167016, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 73741, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 73741, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 93275, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 12290, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 230, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 102, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 102, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 128, 2024-05-09T08:55:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 8, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 17, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 142446601, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 314928323, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 50291, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 302.254822569, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006010117 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-05-09T08:55:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 21, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 34241, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 34220, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 20, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 95, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 75, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 322, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 63935, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 63613, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 1, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 37097, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 37096, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 74500, 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223612 2024-05-09T08:55:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 302080, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 9, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 210086, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 5299, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 18407, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 34094, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 175.981670424, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005161661 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2823, 2024-05-09T08:55:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 22.260916750, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.007885553 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 59466, 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 73.721276728, 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001239721 2024-05-09T08:55:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000245725, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000245725 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-05-09T08:55:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-05-09T08:55:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 53704, 2024-05-09T08:55:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 231.429177721, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004309347 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7345, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 43.006757497, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005855242 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-05-09T08:55:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 9, 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.076718385, 2024-05-09T08:55:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.008524265 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-05-09T08:55:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.022939643, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.011469821 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1428, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.547771407, 2024-05-09T08:55:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001083873 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 34173, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 107.489374888, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003145447 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 1, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 20503, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 20503, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 1537, 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-05-09T08:55:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:55:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:55:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 1261788, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 474, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 176, 2024-05-09T08:55:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 2, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7432, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 156, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 741287, 2024-05-09T08:55:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 21809, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 87982, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 317886311, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 27, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 87982, 2024-05-09T08:55:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87982, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 663.820247118, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.007544955 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 87982, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 89516 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 87982, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 989, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 86993, 2024-05-09T08:55:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 95, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 989, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 51280, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 513, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 3, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 34324, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-05-09T08:55:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 190, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-05-09T08:55:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-05-09T08:55:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 88, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 2, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 18, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 6, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 1536, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 586 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 34182, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 34182, 2024-05-09T08:55:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 576660, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 119857615, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 576660, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 576660, 2024-05-09T08:55:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 119857615, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:55:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 87982, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 321338693, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 87982, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 321338693, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 87982, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 87982, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 87982, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 87982, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 34182, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 3452382, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:55:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 34182, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 989, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3452382, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x558ed8d6e500": { 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-05-09T08:55:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 167016, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 310734019, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 167016, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 50291, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 310734019, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:55:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:55:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:55:59.397 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:56:00.325 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-05-09T08:56:00.325 INFO:tasks.check_counter:Daemon mds.a mds.imported=5 2024-05-09T08:56:00.326 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T08:56:00.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T08:56:00.785 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:56:01.116 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:00 smithi133 ceph-mon[29610]: pgmap v1159: 97 pgs: 97 active+clean; 587 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.7 MiB/s wr, 384 op/s 2024-05-09T08:56:01.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:00 smithi160 ceph-mon[34070]: pgmap v1159: 97 pgs: 97 active+clean; 587 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.7 MiB/s wr, 384 op/s 2024-05-09T08:56:01.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:00 smithi188 ceph-mon[34645]: pgmap v1159: 97 pgs: 97 active+clean; 587 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.7 MiB/s wr, 384 op/s 2024-05-09T08:56:02.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:02 smithi133 ceph-mon[29610]: pgmap v1160: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 7.1 MiB/s wr, 310 op/s 2024-05-09T08:56:02.947 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:56:02.947 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:02.947 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2024-05-09T08:56:02.947 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003729665, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000906035, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:02.948 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 464, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 457, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 825792, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 282320, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.142514061, 2024-05-09T08:56:02.949 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.063969814, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.061818671, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.019048175, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 457, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.050692479, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000110924 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 825792, 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 282320 2024-05-09T08:56:02.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 348, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 216928, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.056608913, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.045888111, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.004353903, 2024-05-09T08:56:02.951 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 346, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.038210555, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000110435 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 216928 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-05-09T08:56:02.952 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:02.953 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:02.954 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148744, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 30, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-05-09T08:56:02.955 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-05-09T08:56:02.956 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 462, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 757737, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 462, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 462, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 757737, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:02.958 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:02.958 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:02.958 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:56:03.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:02 smithi160 ceph-mon[34070]: pgmap v1160: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 7.1 MiB/s wr, 310 op/s 2024-05-09T08:56:03.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:02 smithi188 ceph-mon[34645]: pgmap v1160: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 7.1 MiB/s wr, 310 op/s 2024-05-09T08:56:03.756 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T08:56:03.756 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T08:56:03.757 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T08:56:03.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T08:56:04.211 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:56:05.120 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:04 smithi133 ceph-mon[29610]: pgmap v1161: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 6.1 MiB/s wr, 257 op/s 2024-05-09T08:56:05.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:04 smithi160 ceph-mon[34070]: pgmap v1161: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 6.1 MiB/s wr, 257 op/s 2024-05-09T08:56:05.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:04 smithi188 ceph-mon[34645]: pgmap v1161: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 6.1 MiB/s wr, 257 op/s 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 247337, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 351063, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 91069426, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 339123539, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 14, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 74.972789848, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 50.210577464, 2024-05-09T08:56:06.312 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 31.186170379, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.946431504, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 351061, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 40.097341206, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000114217 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 510078, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.204503265, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000400 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 86693488, 2024-05-09T08:56:06.313 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 274679168 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 27890, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 28237, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 7040138, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 95878785, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 8, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 4.993878205, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.724394538, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.924325752, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.945801638, 2024-05-09T08:56:06.314 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 28231, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.667641069, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000129915 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000003063, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000180 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2672, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 695552 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:06.315 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 298473, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 295567, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 76639547, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 144044071, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 43.549719550, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 20.360359663, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 27.167347756, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.500893761, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 295571, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 29.655999120, 2024-05-09T08:56:06.316 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000100334 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 543702, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.171347393, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000315 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 887712, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 286256 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.317 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 76754, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 108.774400915, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001417182 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 36536, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 5.722440405, 2024-05-09T08:56:06.318 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000156624 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 168518, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 147882, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 147882, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 619.684164953, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004190396 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 20636, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 23, 2024-05-09T08:56:06.319 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 16, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 4, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 12, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 16, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 76, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 1, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 1, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 3, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 312259, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 153682, 2024-05-09T08:56:06.320 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 20629, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 30, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 4583, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 5, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 45, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 14, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 115, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:56:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 10, 2024-05-09T08:56:06.322 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 10455, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 81071, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 44732, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 18377, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 9909, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 105849 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 2, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 40, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 29, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 46, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 36, 2024-05-09T08:56:06.323 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 22945, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 22945, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:56:06.324 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 6, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.325 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 126869, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 28878, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 24581, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 102288, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 4297, 2024-05-09T08:56:06.326 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 165, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 37, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 32, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 133, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 9, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 5, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 35161312, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 216470394, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-05-09T08:56:06.327 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 39137, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 225.240860538, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005755189 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 22, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 23004, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 22982, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 21, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 68, 2024-05-09T08:56:06.328 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 47, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 47, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 45340, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 45293, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 1, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 24773, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 24772, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 70640, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223612 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 266337, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 9584, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 168518, 2024-05-09T08:56:06.329 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 3382, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 39665, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 22867, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 163.205698814, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.007137171 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8637, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 103.467542513, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.011979569 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-05-09T08:56:06.330 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 51845, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 62.128093652, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001198343 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-05-09T08:56:06.333 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.334 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 35295, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 135.128921777, 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003828557 2024-05-09T08:56:06.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5396, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 46.902152389, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.008692022 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.336 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 9, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.169826863, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.018869651 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.337 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 529, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.783757818, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005262302 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 368, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.462551923, 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001256934 2024-05-09T08:56:06.338 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 22936, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 105.435619204, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004596948 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-05-09T08:56:06.340 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 3, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 13402, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 13402, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 1545, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:06.341 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:06.342 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:06.343 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:06.344 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 304659, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 128, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 264, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 3, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:06.345 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7432, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 156, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 122030, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 1368, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-05-09T08:56:06.346 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 66663, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 217181239, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 30, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 66663, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 66663, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 490.855270097, 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.007363234 2024-05-09T08:56:06.347 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 66663, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 68907 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 66663, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1186, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 65477, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 41, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1186, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 40323, 2024-05-09T08:56:06.348 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 698, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 23014, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 82, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-05-09T08:56:06.349 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-05-09T08:56:06.350 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 88, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 2, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 2782, 2024-05-09T08:56:06.351 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 781 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-05-09T08:56:06.352 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 22945, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 22945, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 573696, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 112241077, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 573696, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 573696, 2024-05-09T08:56:06.353 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 112241077, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-05-09T08:56:06.354 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 66663, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 219498684, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 66663, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 219498684, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-05-09T08:56:06.355 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 66663, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 66663, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 66663, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 66663, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:56:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 22945, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 2317445, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 22945, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1186, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2317445, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.357 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55a0da9f6500": { 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 126869, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 212276090, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 126869, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 39137, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 212276090, 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:06.358 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:06.359 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:56:07.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:06 smithi160 ceph-mon[34070]: pgmap v1162: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.6 MiB/s wr, 208 op/s 2024-05-09T08:56:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:06 smithi133 ceph-mon[29610]: pgmap v1162: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.6 MiB/s wr, 208 op/s 2024-05-09T08:56:07.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:06 smithi188 ceph-mon[34645]: pgmap v1162: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.6 MiB/s wr, 208 op/s 2024-05-09T08:56:07.254 INFO:tasks.check_counter:Daemon mds.g mds.exported=5 2024-05-09T08:56:07.254 INFO:tasks.check_counter:Daemon mds.g mds.imported=14 2024-05-09T08:56:07.254 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T08:56:07.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T08:56:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.a/config 2024-05-09T08:56:09.107 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:08 smithi133 ceph-mon[29610]: pgmap v1163: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-09T08:56:09.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:08 smithi160 ceph-mon[34070]: pgmap v1163: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-09T08:56:09.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:08 smithi188 ceph-mon[34645]: pgmap v1163: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2024-05-09T08:56:09.980 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003860701, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.001166159, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:09.981 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 467, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 460, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 826608, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 284240, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.141050925, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.065131018, 2024-05-09T08:56:09.982 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.056652406, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.018972649, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 460, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.045810277, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000099587 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 826608, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 284240 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.983 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 351, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 320, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 217456, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.060644458, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.048456384, 2024-05-09T08:56:09.984 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.004123754, 2024-05-09T08:56:09.985 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:09.985 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:09.985 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 349, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.040992091, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000117455 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 217456 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-05-09T08:56:09.986 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:09.988 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148704, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:09.989 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-05-09T08:56:09.990 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 465, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 758127, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 465, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 465, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 758127, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-05-09T08:56:09.991 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:09.992 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-05-09T08:56:09.992 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-05-09T08:56:10.775 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T08:56:10.776 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T08:56:10.776 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T08:56:10.776 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T08:56:11.100 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:56:11.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:10 smithi160 ceph-mon[34070]: pgmap v1164: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 38 op/s 2024-05-09T08:56:11.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:10 smithi133 ceph-mon[29610]: pgmap v1164: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 38 op/s 2024-05-09T08:56:11.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:10 smithi188 ceph-mon[34645]: pgmap v1164: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 38 op/s 2024-05-09T08:56:12.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:12 smithi160 ceph-mon[34070]: pgmap v1165: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-05-09T08:56:13.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:12 smithi133 ceph-mon[29610]: pgmap v1165: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-05-09T08:56:13.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:12 smithi188 ceph-mon[34645]: pgmap v1165: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-05-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:13.316 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 321990, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 316007, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 85014538, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 647344835, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 21.583032090, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 8.830176122, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 9.573362838, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 4.331754618, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 316005, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 11.703613125, 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000037036 2024-05-09T08:56:13.317 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 402626, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.058707518, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000145 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 32816, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 1456 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 938863, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1529822, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 346603469, 2024-05-09T08:56:13.318 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1631405832, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 13, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 61.590400476, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 28.943510512, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 38.606681749, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 3.898159265, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1529822, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 47.908001147, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000031316 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1771128, 2024-05-09T08:56:13.319 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.288847084, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000163 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 319608992, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 1261888896 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 181616, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 203675, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 42446730, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 312783546, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 12.616558287, 2024-05-09T08:56:13.320 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 5.132389312, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 5.219783206, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 2.581835297, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 203673, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 8.968305373, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000044032 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 247119, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.040124834, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000162 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 788240 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.321 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 231167, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 473.647290092, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002048939 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T08:56:13.322 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 197358, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 58.444619055, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000296135 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 603607, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 572348, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 572348, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 4974.415989414, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.008691243 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-09T08:56:13.323 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 31259, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 618, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 50, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 66, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 6969, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 4730, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 2229, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 10, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 200, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 428, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 2, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 2, 2024-05-09T08:56:13.324 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 6, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 790356, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 516605, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 31259, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 34, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 2953, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 17, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 244, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 8, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 48, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-09T08:56:13.325 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 4, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 234, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 20, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 39159, 2024-05-09T08:56:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 448690, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 183703, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 91741, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 40145, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 337237 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 18, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 6, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 4, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 36, 2024-05-09T08:56:13.327 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 61, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 21, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 114020, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 113999, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:56:13.328 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 17, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 116, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 533123, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 428085, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 417861, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 115262, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 10224, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 631, 2024-05-09T08:56:13.329 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 503, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 491, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 140, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 10, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 12, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 891046428, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 1134149823, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 146277, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1437.905080671, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009830014 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.330 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 45, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 114267, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 114222, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 58, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 606, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 548, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 6969, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 213023, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 206054, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 2, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 117474, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 117472, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 97284, 2024-05-09T08:56:13.331 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 231804 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 751281, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 29303, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 603607, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 15390, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 58534, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 114077, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 672.424255674, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005894477 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 22607, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 516.746423510, 2024-05-09T08:56:13.332 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.022857806 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 131680, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 805.895015747, 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006120101 2024-05-09T08:56:13.333 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000298173, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000298173 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.334 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.335 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 119, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.563198995, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004732764 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-09T08:56:13.336 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 10, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.010256169, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001025616 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.337 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 162893, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2084.213617015, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.012794985 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 22993, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 151.090924052, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006571170 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.338 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 106, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 1.031114051, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.009727491 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.339 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 229, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 8.069510058, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.035238035 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-09T08:56:13.340 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3700, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 17.230784839, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.004656968 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.341 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 8, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.022726359, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002840794 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 113925, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 717.117864772, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.006294648 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-09T08:56:13.343 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 3, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 2, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 47661, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 47661, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 1552, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 360, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 360, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:13.344 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:13.345 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:13.346 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 299680, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 133, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:13.347 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7432, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 156, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 5483034, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 79617, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-09T08:56:13.348 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 269558, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 1158774474, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 32, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 269558, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 269558, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2148.169301111, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007969228 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 269558, 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 273887 2024-05-09T08:56:13.349 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 269558, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2111, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 267447, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 47, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 574, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 2111, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 148388, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 1026, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 49, 2024-05-09T08:56:13.350 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 115398, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 1148, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-09T08:56:13.351 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-09T08:56:13.352 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 88, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 3, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 3952, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 1194 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2024-05-09T08:56:13.353 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 10, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 10, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 113999, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 113999, 2024-05-09T08:56:13.354 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 1442465, 2024-05-09T08:56:13.355 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 304177284, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 1442465, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1442465, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 304177284, 2024-05-09T08:56:13.356 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.357 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:13.358 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 269558, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 1170288373, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 269558, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1170288373, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:13.359 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 269558, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 269558, 2024-05-09T08:56:13.360 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 269558, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 269558, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:13.361 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 113999, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 11513899, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 113999, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2111, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 11513899, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.362 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x5615983e0500": { 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 533123, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1129955519, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 533123, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 146277, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1129955519, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:13.363 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:13.364 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.364 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:13.364 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T08:56:14.460 INFO:tasks.check_counter:Daemon mds.b mds.exported=17 2024-05-09T08:56:14.460 INFO:tasks.check_counter:Daemon mds.b mds.imported=8 2024-05-09T08:56:14.461 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T08:56:14.461 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T08:56:14.709 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:56:15.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:14 smithi160 ceph-mon[34070]: pgmap v1166: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-05-09T08:56:15.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:14 smithi133 ceph-mon[29610]: pgmap v1166: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-05-09T08:56:15.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:14 smithi188 ceph-mon[34645]: pgmap v1166: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 0, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 0, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 0, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.001385846, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000407146, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:16.813 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:16.814 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 350, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 320, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 217280, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.014637456, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.011128901, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000788134, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 348, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.019004947, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000054611 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.815 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 217280 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 467, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 460, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 826608, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 284240, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.056885009, 2024-05-09T08:56:16.816 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.020562147, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.026495280, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.009645017, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 460, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.017070301, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000037109 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 826608, 2024-05-09T08:56:16.817 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 284240 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:16.818 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:16.819 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148784, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 31, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:16.820 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:16.821 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 465, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 758127, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 465, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 465, 2024-05-09T08:56:16.822 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 758127, 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:16.823 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T08:56:17.144 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:16 smithi160 ceph-mon[34070]: pgmap v1167: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-09T08:56:17.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:16 smithi133 ceph-mon[29610]: pgmap v1167: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-09T08:56:17.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:16 smithi188 ceph-mon[34645]: pgmap v1167: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-09T08:56:17.695 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-09T08:56:17.695 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-09T08:56:17.695 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T08:56:17.695 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T08:56:17.943 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:56:19.126 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:18 smithi160 ceph-mon[34070]: pgmap v1168: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-09T08:56:19.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:18 smithi133 ceph-mon[29610]: pgmap v1168: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-09T08:56:19.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:18 smithi188 ceph-mon[34645]: pgmap v1168: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4289, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4291, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 118669335, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1363909, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.337045859, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.086197397, 2024-05-09T08:56:19.794 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.133844967, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.095425271, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4289, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.107798735, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000025133 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.795 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1099, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1083, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 84923238, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 481052, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.142934626, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.033792976, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.081843872, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.023571548, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1083, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.037880475, 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000034977 2024-05-09T08:56:19.796 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 879552, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 284784 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 814, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1162, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 109615232, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 1121194, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2024-05-09T08:56:19.797 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.129425149, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.026667206, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.077278558, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.018404483, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1159, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.041660778, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000035945 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.798 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 863296 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 9715, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.230884627, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000023765 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.799 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2024-05-09T08:56:19.800 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 34192, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 10340, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 23852, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 0, 2024-05-09T08:56:19.801 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 19420, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 1, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2024-05-09T08:56:19.802 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2024-05-09T08:56:19.803 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:56:19.804 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:56:19.805 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2024-05-09T08:56:19.806 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evlrg": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 167016, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2024-05-09T08:56:19.807 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 128, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "segmjr": 15, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 142446601, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 314928323, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 314927185, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 167016 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 13, 2024-05-09T08:56:19.808 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 34218, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 34205, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 11, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 71, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 60, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 34192, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 34413, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 221, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 82368, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 215420 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.809 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.810 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.811 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2024-05-09T08:56:19.812 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.813 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2024-05-09T08:56:19.814 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.815 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.816 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.817 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.818 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.819 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.820 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 1, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 0, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 1, 2024-05-09T08:56:19.821 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 3046, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "mdthresh_evicted": 0 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:19.822 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:19.823 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 240171, 2024-05-09T08:56:19.824 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 80, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 7432, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 156, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:19.825 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 24114552, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 136896, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 5717, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 5717, 2024-05-09T08:56:19.826 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5717, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 11.439836503, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.002001020 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5717, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 5718 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 5717, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 5717, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2024-05-09T08:56:19.827 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 3272, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 2443, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:56:19.828 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2024-05-09T08:56:19.829 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 88, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 2, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 11, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2024-05-09T08:56:19.830 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 1, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.831 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed_ops": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 6198, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 312622275, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:19.832 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 6198, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 6198, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 312622275, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:19.833 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 5717, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 310734621, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 5717, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 310734621, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.834 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 5717, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 5717, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 5717, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 5717, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.835 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:19.836 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x5627088ce6e0": { 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:19.837 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:19.838 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T08:56:20.476 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T08:56:20.476 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T08:56:20.476 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T08:56:20.476 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T08:56:20.716 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.b/config 2024-05-09T08:56:21.151 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:20 smithi160 ceph-mon[34070]: pgmap v1169: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:56:21.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:20 smithi133 ceph-mon[29610]: pgmap v1169: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:56:21.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:20 smithi188 ceph-mon[34645]: pgmap v1169: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T08:56:22.494 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 471, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 464, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 827696, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 286800, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.059765952, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.021721098, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.027854128, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.009741112, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 464, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.018455748, 2024-05-09T08:56:22.495 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000039775 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 827696, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 286800 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 354, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 320, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 217984, 2024-05-09T08:56:22.496 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.014576665, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.011178555, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000593254, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 352, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.020808036, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000059113 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:22.497 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 217984 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 0, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 0, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 0, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.000239904, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000141857, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:22.498 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2024-05-09T08:56:22.499 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:22.500 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:22.501 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 148660, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 29, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2024-05-09T08:56:22.502 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:22.503 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 469, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 758647, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 469, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 469, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 758647, 2024-05-09T08:56:22.504 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T08:56:22.505 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T08:56:23.153 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:22 smithi160 ceph-mon[34070]: pgmap v1170: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T08:56:23.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:22 smithi133 ceph-mon[29610]: pgmap v1170: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T08:56:23.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:22 smithi188 ceph-mon[34645]: pgmap v1170: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T08:56:23.217 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-09T08:56:23.217 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-09T08:56:23.218 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T08:56:23.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T08:56:23.467 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:56:25.156 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:24 smithi188 ceph-mon[34645]: pgmap v1171: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:56:25.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:24 smithi160 ceph-mon[34070]: pgmap v1171: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:56:25.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:24 smithi133 ceph-mon[29610]: pgmap v1171: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-09T08:56:25.219 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-09T08:56:25.219 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1659, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1661, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 65900014, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 527626, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.168802775, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.036234118, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.077711413, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.042803395, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1659, 2024-05-09T08:56:25.220 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.049705429, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000029961 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2437, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2423, 2024-05-09T08:56:25.221 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 93646395, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 907947, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 5, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.243323781, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.068544740, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.104050794, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.057954604, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2422, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.070073372, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000028932 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.222 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 877072, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 285616 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2748, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 3095, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 55362998, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1572322, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-05-09T08:56:25.223 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.288498219, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.083220031, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.098782634, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.082947763, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3094, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.117745868, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000038056 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.224 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 700048 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 10998, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.243479453, 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000022138 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.225 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.226 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 22955, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 2564, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 20391, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 19291, 2024-05-09T08:56:25.227 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 1, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-05-09T08:56:25.228 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:56:25.229 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-05-09T08:56:25.230 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:56:25.231 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-05-09T08:56:25.232 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 126869, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 133, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 12, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 35161312, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 216470394, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 216469795, 2024-05-09T08:56:25.233 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 126869 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 13, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 22983, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 22970, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 11, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 46, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 35, 2024-05-09T08:56:25.234 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 22955, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 23140, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 185, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 73804, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-05-09T08:56:25.235 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.236 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.238 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.239 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-05-09T08:56:25.240 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.241 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.242 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.243 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.244 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.245 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.246 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 4579, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:25.247 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:25.248 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:25.249 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 232875, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 78, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:25.250 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7432, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 156, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 16203704, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 91948, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-09T08:56:25.251 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 6359, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 6359, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6359, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 12.749772416, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002004996 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-05-09T08:56:25.252 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 6359, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 6360 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 6359, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 6359, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 3276, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 3081, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-05-09T08:56:25.253 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-05-09T08:56:25.254 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-05-09T08:56:25.255 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 88, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 10, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 11, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 1, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-05-09T08:56:25.256 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-05-09T08:56:25.257 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 6840, 2024-05-09T08:56:25.258 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 214266079, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 6840, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6840, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 214266079, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:25.259 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 6359, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 212279563, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6359, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 212279563, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 6359, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 6359, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6359, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 6359, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.261 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:25.262 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x56248c7906e0": { 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:25.264 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-09T08:56:26.050 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T08:56:26.050 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-09T08:56:26.050 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T08:56:26.050 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T08:56:26.304 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:56:27.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:26 smithi160 ceph-mon[34070]: pgmap v1172: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T08:56:27.170 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:26 smithi188 ceph-mon[34645]: pgmap v1172: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T08:56:27.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:26 smithi133 ceph-mon[29610]: pgmap v1172: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-09T08:56:28.082 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 0, 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2, 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 0, 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 0, 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.001114016, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000274796, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:28.091 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 465, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 460, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 824128, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 285072, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.061494023, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.023964960, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.025965661, 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.010013487, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 459, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.018656298, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000040645 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 824128, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 285072 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:28.093 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 6, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 354, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 3344, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 218080, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.017587470, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.013571961, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000989475, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:28.094 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 353, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.019200949, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000054393 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 218080 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 0, 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 148722, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 30, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 2, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:28.099 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 467, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 758387, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 467, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 467, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 758387, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:28.100 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:28.101 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:28.101 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-09T08:56:28.810 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-05-09T08:56:28.810 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-05-09T08:56:28.810 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T08:56:28.810 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T08:56:29.057 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:56:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:28 smithi160 ceph-mon[34070]: pgmap v1173: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:29.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:28 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:56:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:28 smithi133 ceph-mon[29610]: pgmap v1173: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:29.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:28 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:56:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:28 smithi188 ceph-mon[34645]: pgmap v1173: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:29.179 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:28 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3439, 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 3441, 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 374305735, 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1093612, 2024-05-09T08:56:30.719 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.479683494, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.082038193, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.276943188, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.095295072, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3439, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.106442338, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000030951 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.720 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2085, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2432, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 461911664, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1450324, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-05-09T08:56:30.721 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.381580412, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.051253538, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.268355390, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.047061948, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2431, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.070912963, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000029170 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.722 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 789136 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 2146, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 2132, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 296591520, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 815247, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 5, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.287571453, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.064848352, 2024-05-09T08:56:30.723 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.164330714, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.047181693, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 2131, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.065648540, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000030806 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 877232, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 285776 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.724 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 11728, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.423904574, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000036144 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.725 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-05-09T08:56:30.726 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 114222, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 62113, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 52109, 2024-05-09T08:56:30.727 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 25489, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 2, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-05-09T08:56:30.728 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 4, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 234, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-05-09T08:56:30.729 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-05-09T08:56:30.730 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-05-09T08:56:30.731 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-05-09T08:56:30.732 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 533124, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 140, 2024-05-09T08:56:30.733 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 45, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 891046428, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 1134151475, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 1134149823, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 533124 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 44, 2024-05-09T08:56:30.734 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 114209, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 114165, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 53, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 597, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 544, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 114222, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 115455, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 1233, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 154096, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215420 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-05-09T08:56:30.735 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.736 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.737 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.738 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 2, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 4585, 2024-05-09T08:56:30.750 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:30.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:30.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 241738, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 102, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-09T08:56:30.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 7432, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 156, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 80585231, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 459662, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:30.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 7183, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 26, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 4, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 7183, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7183, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 17.401689470, 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002422621 2024-05-09T08:56:30.755 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 7183, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 7184 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 7183, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 7183, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 3922, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 3258, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-05-09T08:56:30.756 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-05-09T08:56:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-05-09T08:56:30.758 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 88, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 2, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 11, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 11, 2024-05-09T08:56:30.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-05-09T08:56:30.760 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 7666, 2024-05-09T08:56:30.761 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1132091162, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 7666, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 7666, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1132091162, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-05-09T08:56:30.762 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 7183, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 1129960227, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 7183, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1129960227, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.763 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 7183, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 7183, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 7183, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 7183, 2024-05-09T08:56:30.764 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:30.765 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x5575f8f2e6e0": { 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-05-09T08:56:30.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:30.767 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-09T08:56:31.085 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:30 smithi133 ceph-mon[29610]: pgmap v1174: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:56:31.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:30 smithi160 ceph-mon[34070]: pgmap v1174: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:56:31.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:30 smithi188 ceph-mon[34645]: pgmap v1174: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T08:56:31.545 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-09T08:56:31.546 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-09T08:56:31.546 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T08:56:31.546 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc shell --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T08:56:31.786 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/mon.c/config 2024-05-09T08:56:32.927 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:32 smithi133 ceph-mon[29610]: pgmap v1175: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:33.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:32 smithi160 ceph-mon[34070]: pgmap v1175: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:33.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:32 smithi188 ceph-mon[34645]: pgmap v1175: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 4, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 4, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 3024, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 448, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 4, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 0, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.001282621, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.000042780, 2024-05-09T08:56:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.000534530, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000657319, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000219106 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T08:56:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 471, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 466, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 825760, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 288912, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 2, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 1, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.062763875, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.023546084, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.027561900, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.010302305, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 465, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.018339743, 2024-05-09T08:56:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000039440 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 825760, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 288912 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 358, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 480, 2024-05-09T08:56:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 296528, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 5, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 2, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.018483916, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.014090838, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.001102269, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 355, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.020742529, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000058429 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 296528 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 0, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-05-09T08:56:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-05-09T08:56:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-05-09T08:56:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 148682, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 29, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-05-09T08:56:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 928, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 2, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-05-09T08:56:33.590 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 474, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 759194, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 474, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 474, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 759194, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:33.591 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-05-09T08:56:33.592 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-05-09T08:56:34.371 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-09T08:56:34.371 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-09T08:56:34.371 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T08:56:34.380 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T08:56:34.380 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:56:34.380 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:56:34.448 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-05-09T08:56:34.448 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:56:34.530 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 -- 172.21.15.133:0/3379423562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 msgr2=0x7fa3c0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 --2- 172.21.15.133:0/3379423562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0102f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac02a410 comp rx=0 tx=0).stop 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 -- 172.21.15.133:0/3379423562 shutdown_connections 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 --2- 172.21.15.133:0/3379423562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c0103450 0x7fa3c0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 --2- 172.21.15.133:0/3379423562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0102f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 --2- 172.21.15.133:0/3379423562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c01018a0 0x7fa3c0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.757+0000 7fa3c6200640 1 -- 172.21.15.133:0/3379423562 >> 172.21.15.133:0/3379423562 conn(0x7fa3c00fce10 msgr2=0x7fa3c00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:34.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.758+0000 7fa3c6200640 1 -- 172.21.15.133:0/3379423562 shutdown_connections 2024-05-09T08:56:34.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.758+0000 7fa3c6200640 1 -- 172.21.15.133:0/3379423562 wait complete. 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 Processor -- start 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 -- start start 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 0x7fa3c0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 0x7fa3c019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d500 con 0x7fa3c0102a90 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d380 con 0x7fa3c01018a0 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3c6200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c010d680 con 0x7fa3c0103450 2024-05-09T08:56:34.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3bea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3bea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52074/0 (socket says 172.21.15.133:52074) 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.759+0000 7fa3bea00640 1 -- 172.21.15.133:0/282782539 learned_addr learned my addr 172.21.15.133:0/282782539 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bf400640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 0x7fa3c0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bfe00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 0x7fa3c019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 -- 172.21.15.133:0/282782539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 msgr2=0x7fa3c019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 0x7fa3c019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 -- 172.21.15.133:0/282782539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 msgr2=0x7fa3c0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 0x7fa3c0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 -- 172.21.15.133:0/282782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019f830 con 0x7fa3c0102a90 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bfe00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 0x7fa3c019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bf400640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 0x7fa3c0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.760+0000 7fa3bea00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009230 tx=0x7fa3ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:34.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.761+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac03f030 con 0x7fa3c0102a90 2024-05-09T08:56:34.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.761+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3ac003ce0 con 0x7fa3c0102a90 2024-05-09T08:56:34.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.761+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019fac0 con 0x7fa3c0102a90 2024-05-09T08:56:34.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.761+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac035070 con 0x7fa3c0102a90 2024-05-09T08:56:34.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.761+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c019ffb0 con 0x7fa3c0102a90 2024-05-09T08:56:34.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.762+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0108ce0 con 0x7fa3c0102a90 2024-05-09T08:56:34.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.765+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa3ac042030 con 0x7fa3c0102a90 2024-05-09T08:56:34.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.767+0000 7fa3b7e00640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 0x7fa3a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:34.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.767+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa3ac0ef520 con 0x7fa3c0102a90 2024-05-09T08:56:34.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.767+0000 7fa3bf400640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 0x7fa3a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:34.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.768+0000 7fa3bf400640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 0x7fa3a40a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa3c01990f0 tx=0x7fa3b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:34.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.772+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3ac0bc4b0 con 0x7fa3c0102a90 2024-05-09T08:56:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.980+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c0062430 con 0x7fa3c0102a90 2024-05-09T08:56:34.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.983+0000 7fa3b7e00640 1 -- 172.21.15.133:0/282782539 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7fa3ac0bbdb0 con 0x7fa3c0102a90 2024-05-09T08:56:34.983 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:56:34.983 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 msgr2=0x7fa3a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 0x7fa3a40a5050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa3c01990f0 tx=0x7fa3b0005f90 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 msgr2=0x7fa3c0198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009230 tx=0x7fa3ac0042f0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 shutdown_connections 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7fa3a40a2b90 0x7fa3a40a5050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c0103450 0x7fa3c019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c0102a90 0x7fa3c0198160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 --2- 172.21.15.133:0/282782539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3c01018a0 0x7fa3c0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 >> 172.21.15.133:0/282782539 conn(0x7fa3c00fce10 msgr2=0x7fa3c0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:34.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.986+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 shutdown_connections 2024-05-09T08:56:34.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:34.987+0000 7fa3c6200640 1 -- 172.21.15.133:0/282782539 wait complete. 2024-05-09T08:56:35.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:22:41.209849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:27:53.573664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:27:54.749917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:56:35.005 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:56:35.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:34 smithi160 ceph-mon[34070]: pgmap v1176: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:35.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:34 smithi133 ceph-mon[29610]: pgmap v1176: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:35.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:34 smithi188 ceph-mon[34645]: pgmap v1176: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 -- 172.21.15.133:0/1914254908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88101870 msgr2=0x7f5c88101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/1914254908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88101870 0x7f5c88101c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009860 tx=0x7f5c7c02a360 comp rx=0 tx=0).stop 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 -- 172.21.15.133:0/1914254908 shutdown_connections 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/1914254908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88103420 0x7f5c88109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/1914254908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c88102a60 0x7f5c88102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/1914254908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88101870 0x7f5c88101c90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.204+0000 7f5c8fe00640 1 -- 172.21.15.133:0/1914254908 >> 172.21.15.133:0/1914254908 conn(0x7f5c880fcde0 msgr2=0x7f5c880ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.205+0000 7f5c8fe00640 1 -- 172.21.15.133:0/1914254908 shutdown_connections 2024-05-09T08:56:35.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.205+0000 7f5c8fe00640 1 -- 172.21.15.133:0/1914254908 wait complete. 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.205+0000 7f5c8fe00640 1 Processor -- start 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 -- start start 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c88102a60 0x7f5c881981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 0x7f5c8819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810d560 con 0x7f5c88101870 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810d3e0 con 0x7f5c88102a60 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8810d6e0 con 0x7f5c88103420 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52092/0 (socket says 172.21.15.133:52092) 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8d400640 1 -- 172.21.15.133:0/2027845581 learned_addr learned my addr 172.21.15.133:0/2027845581 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.206+0000 7f5c8de00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 0x7f5c8819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 -- 172.21.15.133:0/2027845581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 msgr2=0x7f5c8819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 0x7f5c8819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 -- 172.21.15.133:0/2027845581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c88102a60 msgr2=0x7f5c881981c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c88102a60 0x7f5c881981c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 -- 172.21.15.133:0/2027845581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8819f890 con 0x7f5c88101870 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8de00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 0x7f5c8819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:35.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8d400640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0304b0 tx=0x7f5c7c032830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c037040 con 0x7f5c88101870 2024-05-09T08:56:35.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7c004050 con 0x7f5c88101870 2024-05-09T08:56:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8819fb20 con 0x7f5c88101870 2024-05-09T08:56:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.207+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c005580 con 0x7f5c88101870 2024-05-09T08:56:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.208+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c881a0010 con 0x7f5c88101870 2024-05-09T08:56:35.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.209+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c88108c00 con 0x7f5c88101870 2024-05-09T08:56:35.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.211+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5c7c004200 con 0x7f5c88101870 2024-05-09T08:56:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.213+0000 7f5c86000640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 0x7f5c700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.213+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5c7c0eee70 con 0x7f5c88101870 2024-05-09T08:56:35.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.213+0000 7f5c8ca00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 0x7f5c700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.214+0000 7f5c8ca00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 0x7f5c700a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009670 tx=0x7f5c74006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.218+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c7c0bbe00 con 0x7f5c88101870 2024-05-09T08:56:35.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.415+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5c880624d0 con 0x7f5c88101870 2024-05-09T08:56:35.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.416+0000 7f5c86000640 1 -- 172.21.15.133:0/2027845581 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f5c7c0bb700 con 0x7f5c88101870 2024-05-09T08:56:35.415 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 msgr2=0x7f5c700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 0x7f5c700a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009670 tx=0x7f5c74006bd0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 msgr2=0x7f5c88197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0304b0 tx=0x7f5c7c032830 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 shutdown_connections 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f5c700a2b90 0x7f5c700a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c88103420 0x7f5c8819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c88102a60 0x7f5c881981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2027845581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c88101870 0x7f5c88197c80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 >> 172.21.15.133:0/2027845581 conn(0x7f5c880fcde0 msgr2=0x7f5c88105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.418+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 shutdown_connections 2024-05-09T08:56:35.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.419+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2027845581 wait complete. 2024-05-09T08:56:35.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:56:35.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.629+0000 7ff99f200640 1 -- 172.21.15.133:0/1239695164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100030 msgr2=0x7ff998100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.629+0000 7ff99f200640 1 --2- 172.21.15.133:0/1239695164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100030 0x7ff998100450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009900 tx=0x7ff98c02a480 comp rx=0 tx=0).stop 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 -- 172.21.15.133:0/1239695164 shutdown_connections 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 --2- 172.21.15.133:0/1239695164 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998101330 0x7ff9981017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 --2- 172.21.15.133:0/1239695164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998100990 0x7ff998100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 --2- 172.21.15.133:0/1239695164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100030 0x7ff998100450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 -- 172.21.15.133:0/1239695164 >> 172.21.15.133:0/1239695164 conn(0x7ff9980fbbd0 msgr2=0x7ff9980fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 -- 172.21.15.133:0/1239695164 shutdown_connections 2024-05-09T08:56:35.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.630+0000 7ff99f200640 1 -- 172.21.15.133:0/1239695164 wait complete. 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 Processor -- start 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 -- start start 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100990 0x7ff99819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998101330 0x7ff9981a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810f5e0 con 0x7ff998100990 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810f460 con 0x7ff998100030 2024-05-09T08:56:35.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.631+0000 7ff99f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810f760 con 0x7ff998101330 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:55744/0 (socket says 172.21.15.133:55744) 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 -- 172.21.15.133:0/2661526908 learned_addr learned my addr 172.21.15.133:0/2661526908 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99e800640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998101330 0x7ff9981a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99d400640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100990 0x7ff99819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 -- 172.21.15.133:0/2661526908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998101330 msgr2=0x7ff9981a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998101330 0x7ff9981a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 -- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100990 msgr2=0x7ff99819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100990 0x7ff99819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 -- 172.21.15.133:0/2661526908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9981a17e0 con 0x7ff998100030 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.632+0000 7ff99de00640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff98c031440 tx=0x7ff98c032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.633+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c03f030 con 0x7ff998100030 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.633+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c003ce0 con 0x7ff998100030 2024-05-09T08:56:35.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.633+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981a1a70 con 0x7ff998100030 2024-05-09T08:56:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.633+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c035070 con 0x7ff998100030 2024-05-09T08:56:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.633+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a1fb0 con 0x7ff998100030 2024-05-09T08:56:35.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.634+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99810ad80 con 0x7ff998100030 2024-05-09T08:56:35.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.637+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff98c042030 con 0x7ff998100030 2024-05-09T08:56:35.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.638+0000 7ff992a00640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.638+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff98c0ef670 con 0x7ff998100030 2024-05-09T08:56:35.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.639+0000 7ff99d400640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.639+0000 7ff99d400640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 0x7ff9800a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff988005e10 tx=0x7ff988005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.640+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff98c0bc5a0 con 0x7ff998100030 2024-05-09T08:56:35.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.843+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff998062430 con 0x7ff998100030 2024-05-09T08:56:35.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.844+0000 7ff992a00640 1 -- 172.21.15.133:0/2661526908 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ff98c0bbea0 con 0x7ff998100030 2024-05-09T08:56:35.843 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 msgr2=0x7ff9800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 0x7ff9800a50d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff988005e10 tx=0x7ff988005da0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 msgr2=0x7ff998199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff98c031440 tx=0x7ff98c032e20 comp rx=0 tx=0).stop 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 shutdown_connections 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff998101330 0x7ff9981a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff998100990 0x7ff99819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 --2- 172.21.15.133:0/2661526908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff998100030 0x7ff998199bc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.846+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 >> 172.21.15.133:0/2661526908 conn(0x7ff9980fbbd0 msgr2=0x7ff9980fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.847+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 shutdown_connections 2024-05-09T08:56:35.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:35.847+0000 7ff99f200640 1 -- 172.21.15.133:0/2661526908 wait complete. 2024-05-09T08:56:35.865 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:56:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 -- 172.21.15.133:0/3824931650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 msgr2=0x7f8470109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 --2- 172.21.15.133:0/3824931650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f8470109ba0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8468009860 tx=0x7f846802a340 comp rx=0 tx=0).stop 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 -- 172.21.15.133:0/3824931650 shutdown_connections 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 --2- 172.21.15.133:0/3824931650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f8470109ba0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 --2- 172.21.15.133:0/3824931650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f8470102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 --2- 172.21.15.133:0/3824931650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 -- 172.21.15.133:0/3824931650 >> 172.21.15.133:0/3824931650 conn(0x7f84700fccd0 msgr2=0x7f84700ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 -- 172.21.15.133:0/3824931650 shutdown_connections 2024-05-09T08:56:36.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.060+0000 7f8475000640 1 -- 172.21.15.133:0/3824931650 wait complete. 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 Processor -- start 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 -- start start 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f84701a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f847010d3e0 con 0x7f8470102950 2024-05-09T08:56:36.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f847010d260 con 0x7f8470101760 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.061+0000 7f8475000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f847010d560 con 0x7f8470103310 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52120/0 (socket says 172.21.15.133:52120) 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846ea00640 1 -- 172.21.15.133:0/1513903383 learned_addr learned my addr 172.21.15.133:0/1513903383 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846fe00640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f84701a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 -- 172.21.15.133:0/1513903383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 msgr2=0x7f84701a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f84701a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 -- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 msgr2=0x7f847019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 -- 172.21.15.133:0/1513903383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84701a1850 con 0x7f8470101760 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846ea00640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f846f400640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470199c40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8460005fd0 tx=0x7f846000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.062+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846000cc90 con 0x7f8470101760 2024-05-09T08:56:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.063+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84701a1b40 con 0x7f8470101760 2024-05-09T08:56:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.063+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846000ee70 con 0x7f8470101760 2024-05-09T08:56:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.063+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460010430 con 0x7f8470101760 2024-05-09T08:56:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.063+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84701a2040 con 0x7f8470101760 2024-05-09T08:56:36.064 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.064+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84701a2210 con 0x7f8470101760 2024-05-09T08:56:36.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.066+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8460003f40 con 0x7f8470101760 2024-05-09T08:56:36.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.067+0000 7f8467e00640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 0x7f84500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.067+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f84600cf340 con 0x7f8470101760 2024-05-09T08:56:36.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.067+0000 7f846ea00640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 0x7f84500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.068+0000 7f846ea00640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 0x7f84500a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f845800f7d0 tx=0x7f8458009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.071+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f846009c270 con 0x7f8470101760 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/282782539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2027845581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:56:36.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.163 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:35 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2661526908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/282782539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2027845581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:56:36.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:56:36.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.178 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:35 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2661526908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/282782539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2027845581' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' 2024-05-09T08:56:36.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:35 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2661526908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T08:56:36.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.274+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f84701a23e0 con 0x7f8470101760 2024-05-09T08:56:36.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.275+0000 7f8467e00640 1 -- 172.21.15.133:0/1513903383 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f846009bb70 con 0x7f8470101760 2024-05-09T08:56:36.278 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 msgr2=0x7f84500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 0x7f84500a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f845800f7d0 tx=0x7f8458009500 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 msgr2=0x7f8470199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470199c40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8460005fd0 tx=0x7f846000ec20 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 shutdown_connections 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84500a2b90 0x7f84500a5050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8470103310 0x7f84701a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8470102950 0x7f847019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 --2- 172.21.15.133:0/1513903383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8470101760 0x7f8470199c40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 >> 172.21.15.133:0/1513903383 conn(0x7f84700fccd0 msgr2=0x7f8470073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 shutdown_connections 2024-05-09T08:56:36.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.279+0000 7f8475000640 1 -- 172.21.15.133:0/1513903383 wait complete. 2024-05-09T08:56:36.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 -- 172.21.15.133:0/2593881378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 msgr2=0x7f4c7c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 --2- 172.21.15.133:0/2593881378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c105120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009900 tx=0x7f4c6402a3f0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 -- 172.21.15.133:0/2593881378 shutdown_connections 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 --2- 172.21.15.133:0/2593881378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c105660 0x7f4c7c10bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 --2- 172.21.15.133:0/2593881378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c105120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 --2- 172.21.15.133:0/2593881378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 -- 172.21.15.133:0/2593881378 >> 172.21.15.133:0/2593881378 conn(0x7f4c7c0ff020 msgr2=0x7f4c7c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 -- 172.21.15.133:0/2593881378 shutdown_connections 2024-05-09T08:56:36.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.500+0000 7f4c81400640 1 -- 172.21.15.133:0/2593881378 wait complete. 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 Processor -- start 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 -- start start 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c105660 0x7f4c7c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f730 con 0x7f4c7c103ab0 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.501+0000 7f4c81400640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f5b0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c81400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c10f8b0 con 0x7f4c7c105660 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c7a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c7a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52144/0 (socket says 172.21.15.133:52144) 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c7a800640 1 -- 172.21.15.133:0/1676996325 learned_addr learned my addr 172.21.15.133:0/1676996325 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c7b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c105660 0x7f4c7c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 -- 172.21.15.133:0/1676996325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c105660 msgr2=0x7f4c7c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c105660 0x7f4c7c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 -- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 msgr2=0x7f4c7c199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.502+0000 7f4c73e00640 1 -- 172.21.15.133:0/1676996325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1a1a30 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c73e00640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c19a350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c64031440 tx=0x7f4c640043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6403f070 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6403b460 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64035070 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c1a1cc0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.503+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a2160 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.504+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7c10ae50 con 0x7f4c7c104ca0 2024-05-09T08:56:36.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.506+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4c6403b980 con 0x7f4c7c104ca0 2024-05-09T08:56:36.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.507+0000 7f4c73400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 0x7f4c600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.508+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4c640ef5f0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.508+0000 7f4c7a800640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 0x7f4c600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.508+0000 7f4c7a800640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 0x7f4c600a50d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005fd0 tx=0x7f4c6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.512+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4c640bbed0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.725+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c7c0624d0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.726+0000 7f4c73400640 1 -- 172.21.15.133:0/1676996325 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+598 (secure 0 0 0) 0x7f4c640c01d0 con 0x7f4c7c104ca0 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/3643547543 2024-05-10T08:19:56.515544+0000 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/3643547543 2024-05-10T08:19:56.515544+0000 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/899667592 2024-05-10T08:19:56.515544+0000 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2316286583 2024-05-10T08:19:56.515544+0000 2024-05-09T08:56:36.725 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/1186224387 2024-05-10T08:19:22.399959+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/74425247 2024-05-10T08:19:22.399959+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/1186224387 2024-05-10T08:19:22.399959+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/775453271 2024-05-10T08:19:22.399959+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2601959981 2024-05-10T08:19:22.399959+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/3973005658 2024-05-10T08:19:56.515544+0000 2024-05-09T08:56:36.726 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2024-05-09T08:56:36.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 msgr2=0x7f4c600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 0x7f4c600a50d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005fd0 tx=0x7f4c6c00b410 comp rx=0 tx=0).stop 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 msgr2=0x7f4c7c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c19a350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4c64031440 tx=0x7f4c640043b0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 shutdown_connections 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4c600a2c10 0x7f4c600a50d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7c105660 0x7f4c7c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7c104ca0 0x7f4c7c19a350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 --2- 172.21.15.133:0/1676996325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c7c103ab0 0x7f4c7c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.728+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 >> 172.21.15.133:0/1676996325 conn(0x7f4c7c0ff020 msgr2=0x7f4c7c108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.729+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 shutdown_connections 2024-05-09T08:56:36.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:36.729+0000 7f4c81400640 1 -- 172.21.15.133:0/1676996325 wait complete. 2024-05-09T08:56:36.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T08:56:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:36.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:36.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T08:56:36.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:36.967 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T08:56:37.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.069 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-05-09T08:56:37.145 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T08:56:37.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.146 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:56:37.179 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:36 smithi133 ceph-mon[29610]: pgmap v1177: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-09T08:56:37.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:36 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1513903383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:56:37.180 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:36 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/1676996325' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T08:56:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:36 smithi188 ceph-mon[34645]: pgmap v1177: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-09T08:56:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:36 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1513903383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:56:37.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:36 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/1676996325' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T08:56:37.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-05-09T08:56:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:36 smithi160 ceph-mon[34070]: pgmap v1177: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-09T08:56:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:36 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1513903383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T08:56:37.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:36 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/1676996325' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T08:56:37.474 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T08:56:37.485 INFO:tasks.cephadm:Teardown begin 2024-05-09T08:56:37.485 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T08:56:37.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 -- 172.21.15.133:0/4110281743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660100770 msgr2=0x7f3660100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 --2- 172.21.15.133:0/4110281743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660100770 0x7f3660100b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3650009900 tx=0x7f365002a3f0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 -- 172.21.15.133:0/4110281743 shutdown_connections 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 --2- 172.21.15.133:0/4110281743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 0x7f3660108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 --2- 172.21.15.133:0/4110281743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660101960 0x7f3660101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 --2- 172.21.15.133:0/4110281743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660100770 0x7f3660100b90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 -- 172.21.15.133:0/4110281743 >> 172.21.15.133:0/4110281743 conn(0x7f36600fbce0 msgr2=0x7f36600fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 -- 172.21.15.133:0/4110281743 shutdown_connections 2024-05-09T08:56:37.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.681+0000 7f3667200640 1 -- 172.21.15.133:0/4110281743 wait complete. 2024-05-09T08:56:37.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.682+0000 7f3667200640 1 Processor -- start 2024-05-09T08:56:37.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.682+0000 7f3667200640 1 -- start start 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.682+0000 7f3667200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660100770 0x7f3660199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.682+0000 7f3667200640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f3667200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 0x7f36601a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f3667200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f366010c430 con 0x7f3660102320 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f3667200640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f366010c2b0 con 0x7f3660101960 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f3667200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f366010c5b0 con 0x7f3660100770 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:55810/0 (socket says 172.21.15.133:55810) 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 -- 172.21.15.133:0/3671029827 learned_addr learned my addr 172.21.15.133:0/3671029827 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365fe00640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660100770 0x7f3660199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f3665200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 0x7f36601a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 -- 172.21.15.133:0/3671029827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660100770 msgr2=0x7f3660199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660100770 0x7f3660199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 -- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 msgr2=0x7f36601a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 0x7f36601a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 -- 172.21.15.133:0/3671029827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36601a1a20 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.683+0000 7f365f400640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e9f0 tx=0x7f364c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.684+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c00cd80 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.684+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f364c005500 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.684+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36601a1d10 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.684+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c010690 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.684+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36601a2200 con 0x7f3660101960 2024-05-09T08:56:37.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.685+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36601a2490 con 0x7f3660101960 2024-05-09T08:56:37.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.687+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f364c0046c0 con 0x7f3660101960 2024-05-09T08:56:37.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.688+0000 7f365cc00640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.689+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f364c0cf3f0 con 0x7f3660101960 2024-05-09T08:56:37.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.689+0000 7f365fe00640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.689+0000 7f365fe00640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 0x7f36440a50d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f365002a9e0 tx=0x7f36500315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:37.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.692+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f364c09c320 con 0x7f3660101960 2024-05-09T08:56:37.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.922+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36601a2660 con 0x7f3660101960 2024-05-09T08:56:37.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.923+0000 7f365cc00640 1 -- 172.21.15.133:0/3671029827 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f364c086070 con 0x7f3660101960 2024-05-09T08:56:37.923 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:56:37.923 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:55:52.502604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:56:37.923 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 31 2024-05-09T08:56:37.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.925+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 msgr2=0x7f36440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.925+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 0x7f36440a50d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f365002a9e0 tx=0x7f36500315b0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 msgr2=0x7f366019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f364c00e9f0 tx=0x7f364c00edf0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 shutdown_connections 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f36440a2c10 0x7f36440a50d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3660102320 0x7f36601a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3660101960 0x7f366019a340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 --2- 172.21.15.133:0/3671029827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3660100770 0x7f3660199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 >> 172.21.15.133:0/3671029827 conn(0x7f36600fbce0 msgr2=0x7f36600fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 shutdown_connections 2024-05-09T08:56:37.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:37.926+0000 7f3667200640 1 -- 172.21.15.133:0/3671029827 wait complete. 2024-05-09T08:56:37.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.142+0000 7f4dca000640 1 -- 172.21.15.133:0/3124004177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4104aa0 msgr2=0x7f4dc4104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.142+0000 7f4dca000640 1 --2- 172.21.15.133:0/3124004177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4104aa0 0x7f4dc4104f20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009900 tx=0x7f4db002a410 comp rx=0 tx=0).stop 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 -- 172.21.15.133:0/3124004177 shutdown_connections 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 --2- 172.21.15.133:0/3124004177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc4105460 0x7f4dc410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 --2- 172.21.15.133:0/3124004177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4104aa0 0x7f4dc4104f20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 --2- 172.21.15.133:0/3124004177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc41038b0 0x7f4dc4103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 -- 172.21.15.133:0/3124004177 >> 172.21.15.133:0/3124004177 conn(0x7f4dc40fee40 msgr2=0x7f4dc4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 -- 172.21.15.133:0/3124004177 shutdown_connections 2024-05-09T08:56:38.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.143+0000 7f4dca000640 1 -- 172.21.15.133:0/3124004177 wait complete. 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 Processor -- start 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 -- start start 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc4104aa0 0x7f4dc419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4105460 0x7f4dc41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410f580 con 0x7f4dc4105460 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410f400 con 0x7f4dc4104aa0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dca000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410f700 con 0x7f4dc41038b0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dc8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dc8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:55374/0 (socket says 172.21.15.133:55374) 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dc8c00640 1 -- 172.21.15.133:0/3031124549 learned_addr learned my addr 172.21.15.133:0/3031124549 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dc3e00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc4104aa0 0x7f4dc419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.144+0000 7f4dc9600640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4105460 0x7f4dc41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 -- 172.21.15.133:0/3031124549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc4104aa0 msgr2=0x7f4dc419a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc4104aa0 0x7f4dc419a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 -- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4105460 msgr2=0x7f4dc41a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4105460 0x7f4dc41a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 -- 172.21.15.133:0/3031124549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc41a17e0 con 0x7f4dc41038b0 2024-05-09T08:56:38.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc8c00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4db800e9c0 tx=0x7f4db800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db800cd80 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db80035f0 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8010690 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc41a1ad0 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.145+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc41a1fa0 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.146+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc410abd0 con 0x7f4dc41038b0 2024-05-09T08:56:38.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.148+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4db80037a0 con 0x7f4dc41038b0 2024-05-09T08:56:38.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.149+0000 7f4dc1600640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.150+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4db80cf450 con 0x7f4dc41038b0 2024-05-09T08:56:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.150+0000 7f4dc3e00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.150+0000 7f4dc3e00640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4db0031440 tx=0x7f4db00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:38.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.154+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4db809c380 con 0x7f4dc41038b0 2024-05-09T08:56:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.375+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dc402d1c0 con 0x7f4dc41038b0 2024-05-09T08:56:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.376+0000 7f4dc1600640 1 -- 172.21.15.133:0/3031124549 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f4db809bc80 con 0x7f4dc41038b0 2024-05-09T08:56:38.376 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:56:38.376 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1799489914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1799489914},{"type":"v1","addr":"172.21.15.133:6835","nonce":1799489914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14394,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3539634696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3539634696},{"type":"v1","addr":"172.21.15.133:6837","nonce":3539634696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1493326462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1493326462},{"type":"v1","addr":"172.21.15.188:6833","nonce":1493326462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/4109875508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":4109875508},{"type":"v1","addr":"172.21.15.160:6833","nonce":4109875508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/4102808369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4102808369},{"type":"v1","addr":"172.21.15.160:6835","nonce":4102808369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/232300149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":232300149},{"type":"v1","addr":"172.21.15.188:6835","nonce":232300149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T08:27:55.241118+0000","modified":"2024-05-09T08:55:52.502604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24566,"mds_1":24556,"mds_2":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6837/3112088995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":3112088995},{"type":"v1","addr":"172.21.15.188:6837","nonce":3112088995}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/1342622403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1342622403},{"type":"v1","addr":"172.21.15.160:6837","nonce":1342622403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6839/2528648460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":2528648460},{"type":"v1","addr":"172.21.15.188:6839","nonce":2528648460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/3807326844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":3807326844},{"type":"v1","addr":"172.21.15.133:6839","nonce":3807326844}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24556":{"gid":24556,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6841/2374052560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":2374052560},{"type":"v1","addr":"172.21.15.133:6841","nonce":2374052560}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6839/145157679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":145157679},{"type":"v1","addr":"172.21.15.160:6839","nonce":145157679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24536,24556,24566]},"id":1}]} 2024-05-09T08:56:38.377 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 31 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 msgr2=0x7f4dac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4db0031440 tx=0x7f4db00315b0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 msgr2=0x7f4dc4199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4db800e9c0 tx=0x7f4db800ee80 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 shutdown_connections 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f4dac0a2c10 0x7f4dac0a50d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dc4105460 0x7f4dc41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dc4104aa0 0x7f4dc419a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 --2- 172.21.15.133:0/3031124549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc41038b0 0x7f4dc4199bc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.379+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 >> 172.21.15.133:0/3031124549 conn(0x7f4dc40fee40 msgr2=0x7f4dc4107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.380+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 shutdown_connections 2024-05-09T08:56:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.380+0000 7f4dca000640 1 -- 172.21.15.133:0/3031124549 wait complete. 2024-05-09T08:56:38.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:37 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3671029827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:38.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:38.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:37 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3671029827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:38.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:37 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3671029827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:38.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.590+0000 7f0037e00640 1 -- 172.21.15.133:0/1858673880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00381016a0 msgr2=0x7f0038101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.590+0000 7f0037e00640 1 --2- 172.21.15.133:0/1858673880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00381016a0 0x7f0038101ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0028009900 tx=0x7f002802a3f0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 -- 172.21.15.133:0/1858673880 shutdown_connections 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 --2- 172.21.15.133:0/1858673880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0038103250 0x7f0038109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 --2- 172.21.15.133:0/1858673880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038102890 0x7f0038102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 --2- 172.21.15.133:0/1858673880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00381016a0 0x7f0038101ac0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 -- 172.21.15.133:0/1858673880 >> 172.21.15.133:0/1858673880 conn(0x7f00380fcc10 msgr2=0x7f00380ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 -- 172.21.15.133:0/1858673880 shutdown_connections 2024-05-09T08:56:38.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.591+0000 7f0037e00640 1 -- 172.21.15.133:0/1858673880 wait complete. 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 Processor -- start 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 -- start start 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00381016a0 0x7f0038199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 0x7f00381a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d360 con 0x7f0038102890 2024-05-09T08:56:38.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d1e0 con 0x7f00381016a0 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0037e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f003810d4e0 con 0x7f0038103250 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f002e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f002e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52192/0 (socket says 172.21.15.133:52192) 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f002e000640 1 -- 172.21.15.133:0/3476373795 learned_addr learned my addr 172.21.15.133:0/3476373795 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.592+0000 7f0036a00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00381016a0 0x7f0038199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f0037400640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 0x7f00381a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 -- 172.21.15.133:0/3476373795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 msgr2=0x7f00381a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 0x7f00381a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 -- 172.21.15.133:0/3476373795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00381016a0 msgr2=0x7f0038199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00381016a0 0x7f0038199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 -- 172.21.15.133:0/3476373795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381a1860 con 0x7f0038102890 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f0037400640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 0x7f00381a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:38.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f002e000640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f002000d770 tx=0x7f002000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020017070 con 0x7f0038102890 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0020005500 con 0x7f0038102890 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381a1b50 con 0x7f0038102890 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.593+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020004e00 con 0x7f0038102890 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.594+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381a2070 con 0x7f0038102890 2024-05-09T08:56:38.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.595+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00381a2240 con 0x7f0038102890 2024-05-09T08:56:38.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.599+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0020004fa0 con 0x7f0038102890 2024-05-09T08:56:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.600+0000 7f003ca00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 0x7f001c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.600+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f00200cead0 con 0x7f0038102890 2024-05-09T08:56:38.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.600+0000 7f0036a00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 0x7f001c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:38.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.601+0000 7f0036a00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 0x7f001c0a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f002802a9e0 tx=0x7f00280315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:38.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.605+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f002009ba00 con 0x7f0038102890 2024-05-09T08:56:38.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.797+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00381a29d0 con 0x7f0038102890 2024-05-09T08:56:38.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.799+0000 7f003ca00640 1 -- 172.21.15.133:0/3476373795 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22672 (secure 0 0 0) 0x7f002009b300 con 0x7f0038102890 2024-05-09T08:56:38.798 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-05-09T08:56:38.799 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"99f031e6-0ddc-11ef-bc98-c7b262605968","created":"2024-05-09T08:18:49.174777+0000","modified":"2024-05-09T08:28:57.810205+0000","last_up_change":"2024-05-09T08:26:58.073636+0000","last_in_change":"2024-05-09T08:26:38.047647+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:22:41.209849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:27:53.573664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:27:54.749917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d560e467-02ea-4321-815b-434186255b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6803","nonce":2872423449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6805","nonce":2872423449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6809","nonce":2872423449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2872423449},{"type":"v1","addr":"172.21.15.133:6807","nonce":2872423449}]},"public_addr":"172.21.15.133:6803/2872423449","cluster_addr":"172.21.15.133:6805/2872423449","heartbeat_back_addr":"172.21.15.133:6809/2872423449","heartbeat_front_addr":"172.21.15.133:6807/2872423449","state":["exists","up"]},{"osd":1,"uuid":"082e4943-4828-4bdd-a221-337d3c05a7ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6801","nonce":2987709381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6803","nonce":2987709381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6807","nonce":2987709381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":2987709381},{"type":"v1","addr":"172.21.15.160:6805","nonce":2987709381}]},"public_addr":"172.21.15.160:6801/2987709381","cluster_addr":"172.21.15.160:6803/2987709381","heartbeat_back_addr":"172.21.15.160:6807/2987709381","heartbeat_front_addr":"172.21.15.160:6805/2987709381","state":["exists","up"]},{"osd":2,"uuid":"228eae91-e83f-4241-ae1f-a4f38890d2db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6801","nonce":2504154225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6803","nonce":2504154225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6807","nonce":2504154225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2504154225},{"type":"v1","addr":"172.21.15.188:6805","nonce":2504154225}]},"public_addr":"172.21.15.188:6801/2504154225","cluster_addr":"172.21.15.188:6803/2504154225","heartbeat_back_addr":"172.21.15.188:6807/2504154225","heartbeat_front_addr":"172.21.15.188:6805/2504154225","state":["exists","up"]},{"osd":3,"uuid":"c20bfe2a-6caf-485b-b3d2-4d661db1ffef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6811","nonce":2675627022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6813","nonce":2675627022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6817","nonce":2675627022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":2675627022},{"type":"v1","addr":"172.21.15.133:6815","nonce":2675627022}]},"public_addr":"172.21.15.133:6811/2675627022","cluster_addr":"172.21.15.133:6813/2675627022","heartbeat_back_addr":"172.21.15.133:6817/2675627022","heartbeat_front_addr":"172.21.15.133:6815/2675627022","state":["exists","up"]},{"osd":4,"uuid":"ed3f3dd8-5a5d-47ba-aa43-4a2998db5c32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6809","nonce":1804660056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6811","nonce":1804660056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6815","nonce":1804660056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1804660056},{"type":"v1","addr":"172.21.15.160:6813","nonce":1804660056}]},"public_addr":"172.21.15.160:6809/1804660056","cluster_addr":"172.21.15.160:6811/1804660056","heartbeat_back_addr":"172.21.15.160:6815/1804660056","heartbeat_front_addr":"172.21.15.160:6813/1804660056","state":["exists","up"]},{"osd":5,"uuid":"41748ba8-d09c-4d93-a815-11ef99a89358","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6809","nonce":485185361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6811","nonce":485185361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6815","nonce":485185361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":485185361},{"type":"v1","addr":"172.21.15.188:6813","nonce":485185361}]},"public_addr":"172.21.15.188:6809/485185361","cluster_addr":"172.21.15.188:6811/485185361","heartbeat_back_addr":"172.21.15.188:6815/485185361","heartbeat_front_addr":"172.21.15.188:6813/485185361","state":["exists","up"]},{"osd":6,"uuid":"cc1a882a-80f7-4af1-954d-4005e2b92111","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6819","nonce":1108533027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6821","nonce":1108533027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6825","nonce":1108533027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1108533027},{"type":"v1","addr":"172.21.15.133:6823","nonce":1108533027}]},"public_addr":"172.21.15.133:6819/1108533027","cluster_addr":"172.21.15.133:6821/1108533027","heartbeat_back_addr":"172.21.15.133:6825/1108533027","heartbeat_front_addr":"172.21.15.133:6823/1108533027","state":["exists","up"]},{"osd":7,"uuid":"3f4a0b43-4191-4999-93e4-929b414f0822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6817","nonce":3423087941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6819","nonce":3423087941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6823","nonce":3423087941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3423087941},{"type":"v1","addr":"172.21.15.160:6821","nonce":3423087941}]},"public_addr":"172.21.15.160:6817/3423087941","cluster_addr":"172.21.15.160:6819/3423087941","heartbeat_back_addr":"172.21.15.160:6823/3423087941","heartbeat_front_addr":"172.21.15.160:6821/3423087941","state":["exists","up"]},{"osd":8,"uuid":"0fcbb1ac-8999-488b-965a-a1d6236162f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6817","nonce":4036931105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6819","nonce":4036931105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6823","nonce":4036931105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":4036931105},{"type":"v1","addr":"172.21.15.188:6821","nonce":4036931105}]},"public_addr":"172.21.15.188:6817/4036931105","cluster_addr":"172.21.15.188:6819/4036931105","heartbeat_back_addr":"172.21.15.188:6823/4036931105","heartbeat_front_addr":"172.21.15.188:6821/4036931105","state":["exists","up"]},{"osd":9,"uuid":"af5f780d-dfa5-4680-a6cf-c821033f7540","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6827","nonce":3729207054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6829","nonce":3729207054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6833","nonce":3729207054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3729207054},{"type":"v1","addr":"172.21.15.133:6831","nonce":3729207054}]},"public_addr":"172.21.15.133:6827/3729207054","cluster_addr":"172.21.15.133:6829/3729207054","heartbeat_back_addr":"172.21.15.133:6833/3729207054","heartbeat_front_addr":"172.21.15.133:6831/3729207054","state":["exists","up"]},{"osd":10,"uuid":"83143e78-aa23-47d6-836a-3fd438f33c0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6825","nonce":993185215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6827","nonce":993185215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6831","nonce":993185215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":993185215},{"type":"v1","addr":"172.21.15.160:6829","nonce":993185215}]},"public_addr":"172.21.15.160:6825/993185215","cluster_addr":"172.21.15.160:6827/993185215","heartbeat_back_addr":"172.21.15.160:6831/993185215","heartbeat_front_addr":"172.21.15.160:6829/993185215","state":["exists","up"]},{"osd":11,"uuid":"7d934675-6b21-4c54-ac51-12593da6e97e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6825","nonce":2170090955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6827","nonce":2170090955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6831","nonce":2170090955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2170090955},{"type":"v1","addr":"172.21.15.188:6829","nonce":2170090955}]},"public_addr":"172.21.15.188:6825/2170090955","cluster_addr":"172.21.15.188:6827/2170090955","heartbeat_back_addr":"172.21.15.188:6831/2170090955","heartbeat_front_addr":"172.21.15.188:6829/2170090955","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:21:49.383587+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:14.225805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:22:38.219241+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:10.802395+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:23:36.486235+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:02.033545+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:24:36.210245+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:02.562210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:25:28.025340+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:02.538537+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:30.219157+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:26:55.946692+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6801/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6800/3643547543":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/899667592":"2024-05-10T08:19:56.515544+0000","172.21.15.133:0/2316286583":"2024-05-10T08:19:56.515544+0000","172.21.15.133:6801/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/74425247":"2024-05-10T08:19:22.399959+0000","172.21.15.133:6800/1186224387":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/775453271":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/2601959981":"2024-05-10T08:19:22.399959+0000","172.21.15.133:0/3973005658":"2024-05-10T08:19:56.515544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.801+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 msgr2=0x7f001c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.801+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 0x7f001c0a50d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f002802a9e0 tx=0x7f00280315b0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.801+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 msgr2=0x7f003819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.801+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f002000d770 tx=0x7f002000dc30 comp rx=0 tx=0).stop 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 shutdown_connections 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f001c0a2c10 0x7f001c0a50d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0038103250 0x7f00381a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0038102890 0x7f003819a1e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 --2- 172.21.15.133:0/3476373795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00381016a0 0x7f0038199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 >> 172.21.15.133:0/3476373795 conn(0x7f00380fcc10 msgr2=0x7f0038073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 shutdown_connections 2024-05-09T08:56:38.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:38.802+0000 7f0037e00640 1 -- 172.21.15.133:0/3476373795 wait complete. 2024-05-09T08:56:38.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.021+0000 7f7043e00640 1 -- 172.21.15.133:0/83425496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70441021c0 msgr2=0x7f70441045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.021+0000 7f7043e00640 1 --2- 172.21.15.133:0/83425496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70441021c0 0x7f70441045b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7034009ca0 tx=0x7f703402a8c0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 -- 172.21.15.133:0/83425496 shutdown_connections 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 --2- 172.21.15.133:0/83425496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f7044109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 --2- 172.21.15.133:0/83425496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7044104af0 0x7f7044106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 --2- 172.21.15.133:0/83425496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70441021c0 0x7f70441045b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 -- 172.21.15.133:0/83425496 >> 172.21.15.133:0/83425496 conn(0x7f70440fbbd0 msgr2=0x7f70440fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 -- 172.21.15.133:0/83425496 shutdown_connections 2024-05-09T08:56:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.022+0000 7f7043e00640 1 -- 172.21.15.133:0/83425496 wait complete. 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 Processor -- start 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 -- start start 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 0x7f7044199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7044104af0 0x7f704419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f70441a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f704410d3c0 con 0x7f7044104af0 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f704410d240 con 0x7f70441021c0 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.023+0000 7f7043e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f704410d540 con 0x7f7044107420 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7042a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 0x7f7044199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f70441a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7042000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7044104af0 0x7f704419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7042a00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 0x7f7044199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:55884/0 (socket says 172.21.15.133:55884) 2024-05-09T08:56:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7042a00640 1 -- 172.21.15.133:0/4226517339 learned_addr learned my addr 172.21.15.133:0/4226517339 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 -- 172.21.15.133:0/4226517339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 msgr2=0x7f7044199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 0x7f7044199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 -- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7044104af0 msgr2=0x7f704419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7044104af0 0x7f704419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 -- 172.21.15.133:0/4226517339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441a1840 con 0x7f7044107420 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.024+0000 7f7043400640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f70441a11a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e970 tx=0x7f703c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.025+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c00ccc0 con 0x7f7044107420 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.025+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c005020 con 0x7f7044107420 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.025+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441a1b30 con 0x7f7044107420 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.025+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a00 con 0x7f7044107420 2024-05-09T08:56:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.025+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441a2030 con 0x7f7044107420 2024-05-09T08:56:39.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.026+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7044108f70 con 0x7f7044107420 2024-05-09T08:56:39.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.028+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f703c0051d0 con 0x7f7044107420 2024-05-09T08:56:39.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.029+0000 7f703b400640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 0x7f70280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.029+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f703c0ceff0 con 0x7f7044107420 2024-05-09T08:56:39.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.029+0000 7f7042a00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 0x7f70280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.030+0000 7f7042a00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 0x7f70280a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7034004af0 tx=0x7f7034004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.034+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f703c09bf60 con 0x7f7044107420 2024-05-09T08:56:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:38 smithi133 ceph-mon[29610]: pgmap v1178: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T08:56:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:38 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3031124549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:38 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3476373795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.258+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7044062430 con 0x7f7044107420 2024-05-09T08:56:39.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.270+0000 7f703b400640 1 -- 172.21.15.133:0/4226517339 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7f703c09b860 con 0x7f7044107420 2024-05-09T08:56:39.270 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T08:56:39.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 msgr2=0x7f70280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 0x7f70280a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7034004af0 tx=0x7f7034004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 msgr2=0x7f70441a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f70441a11a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e970 tx=0x7f703c00ee30 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 shutdown_connections 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f70280a2b90 0x7f70280a5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7044107420 0x7f70441a11a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7044104af0 0x7f704419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 --2- 172.21.15.133:0/4226517339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70441021c0 0x7f7044199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 >> 172.21.15.133:0/4226517339 conn(0x7f70440fbbd0 msgr2=0x7f70441018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 shutdown_connections 2024-05-09T08:56:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.275+0000 7f7043e00640 1 -- 172.21.15.133:0/4226517339 wait complete. 2024-05-09T08:56:39.300 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T08:56:39.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:38 smithi188 ceph-mon[34645]: pgmap v1178: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T08:56:39.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:38 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3031124549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.309 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:38 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3476373795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.358 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:38 smithi160 ceph-mon[34070]: pgmap v1178: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T08:56:39.358 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:38 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3031124549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.358 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:38 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3476373795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 -- 172.21.15.133:0/1842614992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c108fe0 msgr2=0x7f842c109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 --2- 172.21.15.133:0/1842614992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c108fe0 0x7f842c109460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8414009900 tx=0x7f841402a3f0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 -- 172.21.15.133:0/1842614992 shutdown_connections 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 --2- 172.21.15.133:0/1842614992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c1099a0 0x7f842c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 --2- 172.21.15.133:0/1842614992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c108fe0 0x7f842c109460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 --2- 172.21.15.133:0/1842614992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 -- 172.21.15.133:0/1842614992 >> 172.21.15.133:0/1842614992 conn(0x7f842c076e30 msgr2=0x7f842c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:39.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 -- 172.21.15.133:0/1842614992 shutdown_connections 2024-05-09T08:56:39.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.488+0000 7f8433800640 1 -- 172.21.15.133:0/1842614992 wait complete. 2024-05-09T08:56:39.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.489+0000 7f8433800640 1 Processor -- start 2024-05-09T08:56:39.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.489+0000 7f8433800640 1 -- start start 2024-05-09T08:56:39.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.489+0000 7f8433800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8433800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c108fe0 0x7f842c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8433800640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 0x7f842c1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8433800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f842c113b30 con 0x7f842c107e90 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8433800640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f842c1139b0 con 0x7f842c1099a0 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8433800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f842c113cb0 con 0x7f842c108fe0 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8430e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8430e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52268/0 (socket says 172.21.15.133:52268) 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8430e00640 1 -- 172.21.15.133:0/285795848 learned_addr learned my addr 172.21.15.133:0/285795848 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 0x7f842c1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8423e00640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c108fe0 0x7f842c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 -- 172.21.15.133:0/285795848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c108fe0 msgr2=0x7f842c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c108fe0 0x7f842c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 -- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 msgr2=0x7f842c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8431800640 1 -- 172.21.15.133:0/285795848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a5dd0 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.490+0000 7f8430e00640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f8431800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 0x7f842c1a5760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8424009fd0 tx=0x7f842400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842400ce20 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8424005020 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424010720 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a60c0 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.491+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a6590 con 0x7f842c1099a0 2024-05-09T08:56:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.492+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c10f270 con 0x7f842c1099a0 2024-05-09T08:56:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.494+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8424005540 con 0x7f842c1099a0 2024-05-09T08:56:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.495+0000 7f842a000640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.495+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f84240cf6e0 con 0x7f842c1099a0 2024-05-09T08:56:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.496+0000 7f8430e00640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.496+0000 7f8430e00640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 0x7f84100a50d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f841c011550 tx=0x7f841c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.497+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f842409c510 con 0x7f842c1099a0 2024-05-09T08:56:39.662 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.662 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.662 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mds[64466]: -- 172.21.15.160:0/1006544956 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556784e63a40 con 0x556784e36c00 2024-05-09T08:56:39.662 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: starting mds.b at 2024-05-09T08:56:39.662 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.663 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.663 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mds[64153]: -- 172.21.15.160:0/414969193 <== mon.2 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c721ef1a40 con 0x55c721ec4c00 2024-05-09T08:56:39.663 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:56:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: starting mds.h at 2024-05-09T08:56:39.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a[29606]: 2024-05-09T08:56:39.261+0000 7f7998a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T08:56:39.677 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.677 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.677 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mds[70994]: -- 172.21.15.133:0/483390141 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564eb89e7a40 con 0x564eb89bac00 2024-05-09T08:56:39.677 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: starting mds.a at 2024-05-09T08:56:39.678 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g[70597]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.678 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g[70597]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.678 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mds[70606]: -- 172.21.15.133:0/579666333 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a82a3eba40 con 0x55a82a3be800 2024-05-09T08:56:39.678 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:56:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g[70597]: starting mds.g at 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mds[65504]: -- 172.21.15.188:0/1523015719 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a7659da40 con 0x559a76571000 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: starting mds.c at 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: ignoring --setuser ceph since I am not root 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: ignoring --setgroup ceph since I am not root 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mds[65810]: -- 172.21.15.188:0/2253601455 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648f2b61a40 con 0x5648f2b35000 2024-05-09T08:56:39.682 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:56:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: starting mds.i at 2024-05-09T08:56:39.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:39.739+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f842c02d450 con 0x7f842c1099a0 2024-05-09T08:56:40.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.279+0000 7f842a000640 1 -- 172.21.15.133:0/285795848 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f842409be10 con 0x7f842c1099a0 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.285+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 msgr2=0x7f84100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.285+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 0x7f84100a50d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f841c011550 tx=0x7f841c005f90 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.285+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 msgr2=0x7f842c1a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.285+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 0x7f842c1a5760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8424009fd0 tx=0x7f842400ef10 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 shutdown_connections 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f842c1099a0 0x7f842c1a5760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f842c108fe0 0x7f842c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 --2- 172.21.15.133:0/285795848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f842c107e90 0x7f842c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 >> 172.21.15.133:0/285795848 conn(0x7f842c076e30 msgr2=0x7f842c10c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.286+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 shutdown_connections 2024-05-09T08:56:40.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.287+0000 7f8433800640 1 -- 172.21.15.133:0/285795848 wait complete. 2024-05-09T08:56:40.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/4226517339' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T08:56:40.306 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T08:56:40.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T08:56:40.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T08:56:40.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T08:56:40.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/285795848' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.307 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:39 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T08:56:40.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/4226517339' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T08:56:40.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T08:56:40.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T08:56:40.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T08:56:40.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T08:56:40.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/285795848' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:39 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/4226517339' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T08:56:40.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T08:56:40.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T08:56:40.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T08:56:40.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T08:56:40.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T08:56:40.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/285795848' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:39 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 -- 172.21.15.133:0/2363450155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8109920 msgr2=0x7faad81101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 --2- 172.21.15.133:0/2363450155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8109920 0x7faad81101b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faad0009ca0 tx=0x7faad002a8c0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 -- 172.21.15.133:0/2363450155 shutdown_connections 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 --2- 172.21.15.133:0/2363450155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8109920 0x7faad81101b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 --2- 172.21.15.133:0/2363450155 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8108f60 0x7faad81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 --2- 172.21.15.133:0/2363450155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8107d70 0x7faad8108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 -- 172.21.15.133:0/2363450155 >> 172.21.15.133:0/2363450155 conn(0x7faad8076d00 msgr2=0x7faad8079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 -- 172.21.15.133:0/2363450155 shutdown_connections 2024-05-09T08:56:40.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.503+0000 7faadd000640 1 -- 172.21.15.133:0/2363450155 wait complete. 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.504+0000 7faadd000640 1 Processor -- start 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.504+0000 7faadd000640 1 -- start start 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.504+0000 7faadd000640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8107d70 0x7faad819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faadd000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faadd000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8109920 0x7faad81a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faadd000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faad81139f0 con 0x7faad8109920 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faadd000640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faad8113870 con 0x7faad8107d70 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faadd000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faad8113b70 con 0x7faad8108f60 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faad7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8109920 0x7faad81a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.133:55464/0 (socket says 172.21.15.133:55464) 2024-05-09T08:56:40.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 -- 172.21.15.133:0/3278340791 learned_addr learned my addr 172.21.15.133:0/3278340791 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faad7400640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8107d70 0x7faad819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 -- 172.21.15.133:0/3278340791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8107d70 msgr2=0x7faad819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8107d70 0x7faad819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 -- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8109920 msgr2=0x7faad81a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8109920 0x7faad81a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.505+0000 7faacfe00640 1 -- 172.21.15.133:0/3278340791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad81a5bf0 con 0x7faad8108f60 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faacfe00640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faac000e9f0 tx=0x7faac000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac000cd80 con 0x7faad8108f60 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0005500 con 0x7faad8108f60 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad81a5ee0 con 0x7faad8108f60 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0010690 con 0x7faad8108f60 2024-05-09T08:56:40.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.506+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a63a0 con 0x7faad8108f60 2024-05-09T08:56:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.507+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad810f030 con 0x7faad8108f60 2024-05-09T08:56:40.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.509+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faac00046c0 con 0x7faad8108f60 2024-05-09T08:56:40.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.510+0000 7faad4c00640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 0x7faabc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.511+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7faac00cf430 con 0x7faad8108f60 2024-05-09T08:56:40.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.511+0000 7faad7400640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 0x7faabc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.511+0000 7faad7400640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 0x7faabc0a50d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faac8005fd0 tx=0x7faac800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.515+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faac009c260 con 0x7faad8108f60 2024-05-09T08:56:40.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:40.720+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faad8062430 con 0x7faad8108f60 2024-05-09T08:56:41.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.284+0000 7faad4c00640 1 -- 172.21.15.133:0/3278340791 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7faac009bb60 con 0x7faad8108f60 2024-05-09T08:56:41.285 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-05-09T08:56:41.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.290+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 msgr2=0x7faabc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.290+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 0x7faabc0a50d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faac8005fd0 tx=0x7faac800b410 comp rx=0 tx=0).stop 2024-05-09T08:56:41.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.290+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 msgr2=0x7faad819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.290+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faac000e9f0 tx=0x7faac000edf0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 shutdown_connections 2024-05-09T08:56:41.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7faabc0a2c10 0x7faabc0a50d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad8109920 0x7faad81a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faad8108f60 0x7faad819e510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 --2- 172.21.15.133:0/3278340791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faad8107d70 0x7faad819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.291+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 >> 172.21.15.133:0/3278340791 conn(0x7faad8076d00 msgr2=0x7faad810c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.293+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 shutdown_connections 2024-05-09T08:56:41.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.293+0000 7faadd000640 1 -- 172.21.15.133:0/3278340791 wait complete. 2024-05-09T08:56:41.325 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: pgmap v1180: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T08:56:41.325 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T08:56:41.325 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: Cluster is now healthy 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6836/1613697912,v1:172.21.15.188:6837/1613697912] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.188:6838/173350504,v1:172.21.15.188:6839/173350504] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6836/2354088139,v1:172.21.15.160:6838/2354088139] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.160:6837/2729354865,v1:172.21.15.160:6839/2729354865] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6838/1146867541,v1:172.21.15.133:6840/1146867541] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: mds.? [v2:172.21.15.133:6839/887637358,v1:172.21.15.133:6841/887637358] up:boot 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: fsmap 12 up:standby 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:56:41.326 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:56:41.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:56:41.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/3278340791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.327 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:40 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T08:56:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: pgmap v1180: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T08:56:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T08:56:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T08:56:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: Cluster is now healthy 2024-05-09T08:56:41.412 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6836/1613697912,v1:172.21.15.188:6837/1613697912] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.188:6838/173350504,v1:172.21.15.188:6839/173350504] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6836/2354088139,v1:172.21.15.160:6838/2354088139] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.160:6837/2729354865,v1:172.21.15.160:6839/2729354865] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6838/1146867541,v1:172.21.15.133:6840/1146867541] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: mds.? [v2:172.21.15.133:6839/887637358,v1:172.21.15.133:6841/887637358] up:boot 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: fsmap 12 up:standby 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:56:41.413 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:56:41.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:56:41.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/3278340791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.414 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:40 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: pgmap v1180: 97 pgs: 97 active+clean; 585 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T08:56:41.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T08:56:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T08:56:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: Cluster is now healthy 2024-05-09T08:56:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T08:56:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6836/1613697912,v1:172.21.15.188:6837/1613697912] up:boot 2024-05-09T08:56:41.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.188:6838/173350504,v1:172.21.15.188:6839/173350504] up:boot 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6836/2354088139,v1:172.21.15.160:6838/2354088139] up:boot 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.160:6837/2729354865,v1:172.21.15.160:6839/2729354865] up:boot 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6838/1146867541,v1:172.21.15.133:6840/1146867541] up:boot 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: mds.? [v2:172.21.15.133:6839/887637358,v1:172.21.15.133:6841/887637358] up:boot 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: fsmap 12 up:standby 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='mgr.14152 172.21.15.133:0/2240136547' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T08:56:41.434 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/3278340791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.435 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:40 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:41.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.620+0000 7f8a38c00640 1 -- 172.21.15.133:0/464480913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 msgr2=0x7f8a340731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.621+0000 7f8a38c00640 1 --2- 172.21.15.133:0/464480913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a340731d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009ca0 tx=0x7f8a2c02a8b0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 -- 172.21.15.133:0/464480913 shutdown_connections 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 --2- 172.21.15.133:0/464480913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 0x7f8a34111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 --2- 172.21.15.133:0/464480913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a340731d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 --2- 172.21.15.133:0/464480913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a34109410 0x7f8a34072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 -- 172.21.15.133:0/464480913 >> 172.21.15.133:0/464480913 conn(0x7f8a3406e680 msgr2=0x7f8a34070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 -- 172.21.15.133:0/464480913 shutdown_connections 2024-05-09T08:56:41.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.624+0000 7f8a38c00640 1 -- 172.21.15.133:0/464480913 wait complete. 2024-05-09T08:56:41.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.626+0000 7f8a38c00640 1 Processor -- start 2024-05-09T08:56:41.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.627+0000 7f8a38c00640 1 -- start start 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.627+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a34109410 0x7f8a3410e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 0x7f8a341ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a38c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34115b30 con 0x7f8a34109410 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a33400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a33400640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.133:55966/0 (socket says 172.21.15.133:55966) 2024-05-09T08:56:41.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a33400640 1 -- 172.21.15.133:0/2405955432 learned_addr learned my addr 172.21.15.133:0/2405955432 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-05-09T08:56:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341159b0 con 0x7f8a34072d50 2024-05-09T08:56:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a33e00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 0x7f8a341ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.628+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34115cb0 con 0x7f8a3410fb70 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 -- 172.21.15.133:0/2405955432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 msgr2=0x7f8a341ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 0x7f8a341ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 -- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a34109410 msgr2=0x7f8a3410e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a34109410 0x7f8a3410e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a341ae3a0 con 0x7f8a34072d50 2024-05-09T08:56:41.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.629+0000 7f8a33400640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400ea60 tx=0x7f8a2400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.630+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400cde0 con 0x7f8a34072d50 2024-05-09T08:56:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.630+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a240035f0 con 0x7f8a34072d50 2024-05-09T08:56:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.630+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24010670 con 0x7f8a34072d50 2024-05-09T08:56:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.630+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a341ae690 con 0x7f8a34072d50 2024-05-09T08:56:41.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.630+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341aeb60 con 0x7f8a34072d50 2024-05-09T08:56:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.632+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a3419e040 con 0x7f8a34072d50 2024-05-09T08:56:41.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.637+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8a240037a0 con 0x7f8a34072d50 2024-05-09T08:56:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.639+0000 7f8a2be00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 0x7f8a180a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.639+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7944+0+0 (secure 0 0 0) 0x7f8a240cf2b0 con 0x7f8a34072d50 2024-05-09T08:56:41.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.639+0000 7f8a32a00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 0x7f8a180a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.641+0000 7f8a32a00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 0x7f8a180a51a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030860 tx=0x7f8a2c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.642+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a2409c2e0 con 0x7f8a34072d50 2024-05-09T08:56:41.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:41.846+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a34062430 con 0x7f8a34072d50 2024-05-09T08:56:42.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.293+0000 7f8a2be00640 1 -- 172.21.15.133:0/2405955432 <== mon.2 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f8a2409bbe0 con 0x7f8a34072d50 2024-05-09T08:56:42.295 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.302+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 msgr2=0x7f8a180a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.302+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 0x7f8a180a51a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c030860 tx=0x7f8a2c005ef0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.302+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 msgr2=0x7f8a3410e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.302+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400ea60 tx=0x7f8a2400ef20 comp rx=0 tx=0).stop 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 shutdown_connections 2024-05-09T08:56:42.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:6800/2058981411,v1:172.21.15.133:6801/2058981411] conn(0x7f8a180a2ce0 0x7f8a180a51a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3410fb70 0x7f8a341ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a34109410 0x7f8a3410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 --2- 172.21.15.133:0/2405955432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a34072d50 0x7f8a3410e450 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400ea60 tx=0x7f8a2400ef20 comp rx=0 tx=0).stop 2024-05-09T08:56:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.303+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 >> 172.21.15.133:0/2405955432 conn(0x7f8a3406e680 msgr2=0x7f8a3419b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:42.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.304+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 shutdown_connections 2024-05-09T08:56:42.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-05-09T08:56:42.305+0000 7f8a38c00640 1 -- 172.21.15.133:0/2405955432 wait complete. 2024-05-09T08:56:42.341 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:56:42.446 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:56:42.494 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:56:42.533 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T08:56:42.534 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T08:56:42.590 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T08:56:42.605 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T08:56:42.620 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T08:56:42.620 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T08:56:42.620 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a 2024-05-09T08:56:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:42 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T08:56:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:42 smithi160 ceph-mon[34070]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T08:56:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:42 smithi160 ceph-mon[34070]: pgmap v1182: 65 pgs: 65 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:56:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:42 smithi160 ceph-mon[34070]: from='client.? 172.21.15.133:0/2405955432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:42.662 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:42 smithi160 ceph-mon[34070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T08:56:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 ceph-mon[29610]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T08:56:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 ceph-mon[29610]: pgmap v1182: 65 pgs: 65 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:56:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 ceph-mon[29610]: from='client.? 172.21.15.133:0/2405955432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:42.677 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 ceph-mon[29610]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:42 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T08:56:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:42 smithi188 ceph-mon[34645]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T08:56:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:42 smithi188 ceph-mon[34645]: pgmap v1182: 65 pgs: 65 active+clean; 577 KiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T08:56:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:42 smithi188 ceph-mon[34645]: from='client.? 172.21.15.133:0/2405955432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:42.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:42 smithi188 ceph-mon[34645]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T08:56:43.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:42 smithi133 systemd[1]: Stopping Ceph mon.a for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:43.625 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a[29606]: 2024-05-09T08:56:43.186+0000 7f799f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:56:43.625 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a[29606]: 2024-05-09T08:56:43.186+0000 7f799f800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T08:56:43.625 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:43 smithi133 podman[90226]: 2024-05-09 08:56:43.412763989 +0000 UTC m=+0.443234102 container died 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T08:56:44.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:44 smithi133 podman[90226]: 2024-05-09 08:56:44.065093083 +0000 UTC m=+1.095563209 container cleanup 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-96805dc, ceph=True) 2024-05-09T08:56:44.333 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:44 smithi133 bash[90226]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a 2024-05-09T08:56:44.593 INFO:journalctl@ceph.mon.a.smithi133.stdout:May 09 08:56:44 smithi133 podman[90240]: 2024-05-09 08:56:44.332994638 +0000 UTC m=+0.904169561 container remove 84f787f9bd06a51d690ef4c0f6dc6103723ea1de1944494985892fcac65183be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.build-date=20240507) 2024-05-09T08:56:44.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.a.service' 2024-05-09T08:56:44.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:44.759 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T08:56:44.759 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T08:56:44.759 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.b 2024-05-09T08:56:45.162 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:44 smithi160 systemd[1]: Stopping Ceph mon.b for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:46.604 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b[34065]: 2024-05-09T08:56:46.339+0000 7efef8668700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:56:46.604 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b[34065]: 2024-05-09T08:56:46.340+0000 7efef8668700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T08:56:46.912 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:46 smithi160 podman[72112]: 2024-05-09 08:56:46.605286094 +0000 UTC m=+0.410150474 container died a1c25f7a82775c3557c2e0d1ed059654e6a8d0f157fe8731e3da41348e652975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T08:56:47.248 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:47 smithi160 podman[72112]: 2024-05-09 08:56:47.216589294 +0000 UTC m=+1.021453665 container cleanup a1c25f7a82775c3557c2e0d1ed059654e6a8d0f157fe8731e3da41348e652975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T08:56:47.248 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:47 smithi160 bash[72112]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b 2024-05-09T08:56:47.870 INFO:journalctl@ceph.mon.b.smithi160.stdout:May 09 08:56:47 smithi160 podman[72129]: 2024-05-09 08:56:47.568550538 +0000 UTC m=+0.958109718 container remove a1c25f7a82775c3557c2e0d1ed059654e6a8d0f157fe8731e3da41348e652975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T08:56:47.884 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.b.service' 2024-05-09T08:56:48.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:48.141 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T08:56:48.141 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T08:56:48.142 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.c 2024-05-09T08:56:48.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 systemd[1]: Stopping Ceph mon.c for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:48.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c[34635]: 2024-05-09T08:56:48.446+0000 7fbf29fcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:56:48.704 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c[34635]: 2024-05-09T08:56:48.446+0000 7fbf29fcc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T08:56:48.705 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 podman[75297]: 2024-05-09 08:56:48.528773177 +0000 UTC m=+0.185882382 container died 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2) 2024-05-09T08:56:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 podman[75297]: 2024-05-09 08:56:48.914497961 +0000 UTC m=+0.571607164 container cleanup 4b45bd867d114b5e590a271c3bab45671b56074204ea59bb665e5359cdda6578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:56:49.182 INFO:journalctl@ceph.mon.c.smithi188.stdout:May 09 08:56:48 smithi188 bash[75297]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mon-c 2024-05-09T08:56:49.366 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mon.c.service' 2024-05-09T08:56:49.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:49.418 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T08:56:49.418 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T08:56:49.418 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x 2024-05-09T08:56:49.927 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:49 smithi133 systemd[1]: Stopping Ceph mgr.x for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:50.394 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:56:49.991+0000 7ff2ab400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:56:50.394 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x[29837]: 2024-05-09T08:56:49.992+0000 7ff2ab400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-09T08:56:50.394 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:50 smithi133 podman[90347]: 2024-05-09 08:56:50.086141266 +0000 UTC m=+0.235701469 container died a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T08:56:50.677 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:50 smithi133 podman[90347]: 2024-05-09 08:56:50.55257454 +0000 UTC m=+0.702134770 container cleanup a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T08:56:50.677 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:50 smithi133 bash[90347]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x 2024-05-09T08:56:51.177 INFO:journalctl@ceph.mgr.x.smithi133.stdout:May 09 08:56:50 smithi133 podman[90362]: 2024-05-09 08:56:50.829289642 +0000 UTC m=+0.714886967 container remove a1a44a013c1c94e8a764735cc6862b915756f416bc452d88fd18a293caa66bc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:96805dcf909541c4ca9a1e90258de8ae5b6fa1bc, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:56:51.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-09T08:56:51.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:51.464 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T08:56:51.464 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T08:56:51.465 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.y 2024-05-09T08:56:51.805 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:56:51 smithi160 systemd[1]: Stopping Ceph mgr.y for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:52.077 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:56:51 smithi160 podman[74910]: 2024-05-09 08:56:51.80602644 +0000 UTC m=+0.142411087 container died 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T08:56:52.412 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:56:52 smithi160 podman[74910]: 2024-05-09 08:56:52.185953612 +0000 UTC m=+0.522338259 container cleanup 2ad8815f893c4fcc1af0667246713ba0f711c9a8b2b499253990b13af950e8fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-09T08:56:52.412 INFO:journalctl@ceph.mgr.y.smithi160.stdout:May 09 08:56:52 smithi160 bash[74910]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-y 2024-05-09T08:56:52.618 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-09T08:56:52.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:52.653 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T08:56:52.653 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T08:56:52.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.z 2024-05-09T08:56:52.932 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:56:52 smithi188 systemd[1]: Stopping Ceph mgr.z for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:53.260 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:56:52 smithi188 podman[75430]: 2024-05-09 08:56:52.998920467 +0000 UTC m=+0.155673500 container died 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T08:56:53.682 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:56:53 smithi188 podman[75430]: 2024-05-09 08:56:53.378605041 +0000 UTC m=+0.535358063 container cleanup 98f62e89a593785cc21c5ed71f4b69d015c7367bcc17f3db25c4dcff5be5023d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:56:53.682 INFO:journalctl@ceph.mgr.z.smithi188.stdout:May 09 08:56:53 smithi188 bash[75430]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mgr-z 2024-05-09T08:56:53.941 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-09T08:56:53.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:56:53.976 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T08:56:53.976 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T08:56:53.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.0 2024-05-09T08:56:54.381 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:56:54 smithi133 systemd[1]: Stopping Ceph osd.0 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:56:54.927 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:56:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0[44303]: 2024-05-09T08:56:54.663+0000 7f3392a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:56:54.927 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:56:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0[44303]: 2024-05-09T08:56:54.663+0000 7f3392a00700 -1 osd.0 91 *** Got signal Terminated *** 2024-05-09T08:56:54.928 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:56:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0[44303]: 2024-05-09T08:56:54.663+0000 7f3392a00700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:00.778 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:00 smithi133 podman[90468]: 2024-05-09 08:57:00.45195307 +0000 UTC m=+6.017761387 container died 25bfc41f45bc9cc06620b84e03889dc9b388b2b69efedfd42cc873ed80ed9512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T08:57:01.177 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:00 smithi133 podman[90468]: 2024-05-09 08:57:00.899377071 +0000 UTC m=+6.465185398 container cleanup 25bfc41f45bc9cc06620b84e03889dc9b388b2b69efedfd42cc873ed80ed9512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:01.177 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:00 smithi133 bash[90468]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0 2024-05-09T08:57:01.677 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:01 smithi133 podman[90501]: 2024-05-09 08:57:01.201189715 +0000 UTC m=+0.732636133 container remove 25bfc41f45bc9cc06620b84e03889dc9b388b2b69efedfd42cc873ed80ed9512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-09T08:57:02.324 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:01 smithi133 podman[90592]: 2024-05-09 08:57:01.886530804 +0000 UTC m=+0.033128849 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:02.324 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:02 smithi133 podman[90592]: 2024-05-09 08:57:02.018373686 +0000 UTC m=+0.164971763 container create b8c5b7c21e7ae149dbf4f5be5eb64697dd3e4f48762246cd458d4cbab6e4d9f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-09T08:57:02.677 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:02 smithi133 podman[90592]: 2024-05-09 08:57:02.324166501 +0000 UTC m=+0.470764618 container init b8c5b7c21e7ae149dbf4f5be5eb64697dd3e4f48762246cd458d4cbab6e4d9f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T08:57:02.678 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:02 smithi133 podman[90592]: 2024-05-09 08:57:02.336284147 +0000 UTC m=+0.482882194 container start b8c5b7c21e7ae149dbf4f5be5eb64697dd3e4f48762246cd458d4cbab6e4d9f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-09T08:57:02.678 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:02 smithi133 podman[90592]: 2024-05-09 08:57:02.437833622 +0000 UTC m=+0.584431706 container attach b8c5b7c21e7ae149dbf4f5be5eb64697dd3e4f48762246cd458d4cbab6e4d9f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:03.144 INFO:journalctl@ceph.osd.0.smithi133.stdout:May 09 08:57:02 smithi133 podman[90592]: 2024-05-09 08:57:02.799353659 +0000 UTC m=+0.945951756 container died b8c5b7c21e7ae149dbf4f5be5eb64697dd3e4f48762246cd458d4cbab6e4d9f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T08:57:03.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.0.service' 2024-05-09T08:57:03.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:03.686 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T08:57:03.686 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T08:57:03.686 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.1 2024-05-09T08:57:04.023 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:03 smithi160 systemd[1]: Stopping Ceph osd.1 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:04.412 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1[38657]: 2024-05-09T08:57:04.023+0000 7fec1451d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:04.412 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1[38657]: 2024-05-09T08:57:04.023+0000 7fec1451d700 -1 osd.1 91 *** Got signal Terminated *** 2024-05-09T08:57:04.412 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1[38657]: 2024-05-09T08:57:04.023+0000 7fec1451d700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:09.816 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:09 smithi160 podman[75035]: 2024-05-09 08:57:09.505749031 +0000 UTC m=+5.602546761 container died 2ab582a511bb6b276a81fda3aca5faf434305e19f8d0f7054be1c3243eb47810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:57:10.150 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:09 smithi160 podman[75035]: 2024-05-09 08:57:09.924510106 +0000 UTC m=+6.021307814 container cleanup 2ab582a511bb6b276a81fda3aca5faf434305e19f8d0f7054be1c3243eb47810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:10.150 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:09 smithi160 bash[75035]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1 2024-05-09T08:57:10.413 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:10 smithi160 podman[75052]: 2024-05-09 08:57:10.150977363 +0000 UTC m=+0.638018810 container remove 2ab582a511bb6b276a81fda3aca5faf434305e19f8d0f7054be1c3243eb47810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T08:57:10.870 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:10 smithi160 podman[75143]: 2024-05-09 08:57:10.457621607 +0000 UTC m=+0.018202563 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:10.870 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:10 smithi160 podman[75143]: 2024-05-09 08:57:10.602065973 +0000 UTC m=+0.162646902 container create 69c8ebabd44fc77e7f0c3d0c06c63abe4fc7366e3e9692bd076aa708ee44130c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:11.162 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:10 smithi160 podman[75143]: 2024-05-09 08:57:10.872066782 +0000 UTC m=+0.432647713 container init 69c8ebabd44fc77e7f0c3d0c06c63abe4fc7366e3e9692bd076aa708ee44130c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:11.162 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:10 smithi160 podman[75143]: 2024-05-09 08:57:10.87541532 +0000 UTC m=+0.435996254 container start 69c8ebabd44fc77e7f0c3d0c06c63abe4fc7366e3e9692bd076aa708ee44130c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:57:11.163 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:11 smithi160 podman[75143]: 2024-05-09 08:57:11.030172562 +0000 UTC m=+0.590753496 container attach 69c8ebabd44fc77e7f0c3d0c06c63abe4fc7366e3e9692bd076aa708ee44130c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T08:57:11.498 INFO:journalctl@ceph.osd.1.smithi160.stdout:May 09 08:57:11 smithi160 podman[75143]: 2024-05-09 08:57:11.174145051 +0000 UTC m=+0.734725989 container died 69c8ebabd44fc77e7f0c3d0c06c63abe4fc7366e3e9692bd076aa708ee44130c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T08:57:11.832 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.1.service' 2024-05-09T08:57:11.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:11.871 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T08:57:11.871 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T08:57:11.871 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.2 2024-05-09T08:57:12.182 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:11 smithi188 systemd[1]: Stopping Ceph osd.2 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:12.682 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2[39647]: 2024-05-09T08:57:12.188+0000 7ff5ee19d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:12.683 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2[39647]: 2024-05-09T08:57:12.188+0000 7ff5ee19d700 -1 osd.2 91 *** Got signal Terminated *** 2024-05-09T08:57:12.683 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2[39647]: 2024-05-09T08:57:12.188+0000 7ff5ee19d700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:18.005 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:17 smithi188 podman[75558]: 2024-05-09 08:57:17.689171935 +0000 UTC m=+5.620782539 container died c70b2c6547918d38572a6c757b3a914ca954500ac1be07ca19fcdaae695f0b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T08:57:18.372 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75558]: 2024-05-09 08:57:18.105912383 +0000 UTC m=+6.037522976 container cleanup c70b2c6547918d38572a6c757b3a914ca954500ac1be07ca19fcdaae695f0b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:18.373 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 bash[75558]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2 2024-05-09T08:57:18.682 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75575]: 2024-05-09 08:57:18.374199083 +0000 UTC m=+0.676155859 container remove c70b2c6547918d38572a6c757b3a914ca954500ac1be07ca19fcdaae695f0b4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:18.951 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75667]: 2024-05-09 08:57:18.59075377 +0000 UTC m=+0.018195146 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:18.952 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75667]: 2024-05-09 08:57:18.717010445 +0000 UTC m=+0.144451846 container create 69d9b3f6f898ade8c2d1b9c37874f3b655d9ff5b2dc443085be3ef6855ac6325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T08:57:19.290 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75667]: 2024-05-09 08:57:18.953326417 +0000 UTC m=+0.380767783 container init 69d9b3f6f898ade8c2d1b9c37874f3b655d9ff5b2dc443085be3ef6855ac6325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-09T08:57:19.290 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:18 smithi188 podman[75667]: 2024-05-09 08:57:18.9571218 +0000 UTC m=+0.384563156 container start 69d9b3f6f898ade8c2d1b9c37874f3b655d9ff5b2dc443085be3ef6855ac6325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:57:19.290 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:19 smithi188 podman[75667]: 2024-05-09 08:57:19.053514422 +0000 UTC m=+0.480955798 container attach 69d9b3f6f898ade8c2d1b9c37874f3b655d9ff5b2dc443085be3ef6855ac6325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:19.567 INFO:journalctl@ceph.osd.2.smithi188.stdout:May 09 08:57:19 smithi188 podman[75667]: 2024-05-09 08:57:19.291917116 +0000 UTC m=+0.719358475 container died 69d9b3f6f898ade8c2d1b9c37874f3b655d9ff5b2dc443085be3ef6855ac6325 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc) 2024-05-09T08:57:19.979 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.2.service' 2024-05-09T08:57:20.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:20.017 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T08:57:20.017 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T08:57:20.017 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.3 2024-05-09T08:57:20.427 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:20 smithi133 systemd[1]: Stopping Ceph osd.3 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:20.927 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:57:20.615+0000 7f2912400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:20.927 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:57:20.615+0000 7f2912400700 -1 osd.3 91 *** Got signal Terminated *** 2024-05-09T08:57:20.928 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:20 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:57:20.615+0000 7f2912400700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:21.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:21.227+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:22.182 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:21.693+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:22.564 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:22.277+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:22.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:22.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:23.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:23.076+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:23.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:23.312+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:23.667+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:24.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:23 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:23.750+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:24.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:24.051+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:24.664 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:24.336+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:24.884 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:24 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:24.718+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:24.655+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:25.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:25 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:25.050+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:25.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:25.007+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:25.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:25.309+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:25.676 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:25 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3[51227]: 2024-05-09T08:57:25.257+0000 7f2908c00700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:58.888407+0000 front 2024-05-09T08:56:58.888464+0000 (oldest deadline 2024-05-09T08:57:24.788597+0000) 2024-05-09T08:57:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:25.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:26.032 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:25 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:25.706+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:26.393 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:26 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:26.032+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:26.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:26.037+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:26.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:26 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:26.333+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:26.677 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:26 smithi133 podman[90697]: 2024-05-09 08:57:26.394086769 +0000 UTC m=+5.923482602 container died cdfef9052e7dc77483c60457965b95d03238954f108ee9b77aad4b0e077b1945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:27.043 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:26.717+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:27.062 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:26 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:26.727+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:27.062 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:26 smithi133 podman[90697]: 2024-05-09 08:57:26.867495583 +0000 UTC m=+6.396891409 container cleanup cdfef9052e7dc77483c60457965b95d03238954f108ee9b77aad4b0e077b1945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T08:57:27.063 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:26 smithi133 bash[90697]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3 2024-05-09T08:57:27.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:27 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:27.062+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:27.427 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:27 smithi133 podman[90728]: 2024-05-09 08:57:27.102095554 +0000 UTC m=+0.697346146 container remove cdfef9052e7dc77483c60457965b95d03238954f108ee9b77aad4b0e077b1945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T08:57:27.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:27.044+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:27.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:27 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:27.286+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:27.670+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:28.082 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:27 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:27.734+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:28.082 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:27 smithi133 podman[90817]: 2024-05-09 08:57:27.808858049 +0000 UTC m=+0.037329487 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:28.082 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:27 smithi133 podman[90817]: 2024-05-09 08:57:27.944685021 +0000 UTC m=+0.173156469 container create e0c919a088300f137ff063d9f2e122c6243c1ee80d0bb60710e40ddfe2b99496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3-deactivate, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-09T08:57:28.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:28.082+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:28.427 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:28 smithi133 podman[90817]: 2024-05-09 08:57:28.239525166 +0000 UTC m=+0.467996621 container init e0c919a088300f137ff063d9f2e122c6243c1ee80d0bb60710e40ddfe2b99496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, io.buildah.version=1.33.5) 2024-05-09T08:57:28.427 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:28 smithi133 podman[90817]: 2024-05-09 08:57:28.250405222 +0000 UTC m=+0.478876664 container start e0c919a088300f137ff063d9f2e122c6243c1ee80d0bb60710e40ddfe2b99496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=squid-96805dc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:57:28.427 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:28 smithi133 podman[90817]: 2024-05-09 08:57:28.322304366 +0000 UTC m=+0.550775864 container attach e0c919a088300f137ff063d9f2e122c6243c1ee80d0bb60710e40ddfe2b99496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:57:28.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:28.052+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:28.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:28 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:28.265+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:29.002 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:28.705+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:29.021 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:28 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:28.728+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:29.021 INFO:journalctl@ceph.osd.3.smithi133.stdout:May 09 08:57:28 smithi133 podman[90817]: 2024-05-09 08:57:28.697406256 +0000 UTC m=+0.925877711 container died e0c919a088300f137ff063d9f2e122c6243c1ee80d0bb60710e40ddfe2b99496 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-09T08:57:29.384 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:29 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:29.092+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:29.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:29.003+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:29.437 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.3.service' 2024-05-09T08:57:29.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:29.552 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T08:57:29.552 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T08:57:29.552 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.4 2024-05-09T08:57:29.579 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:29 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:29.238+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:29.869 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:29 smithi160 systemd[1]: Stopping Ceph osd.4 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:29.869 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:29 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:29.685+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:30.004 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:29.696+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:30.122 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:29 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:29.715+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:30.162 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:29 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:29.869+0000 7fda83005700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:30.162 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:29 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:29.869+0000 7fda83005700 -1 osd.4 91 *** Got signal Terminated *** 2024-05-09T08:57:30.162 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:29 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:29.869+0000 7fda83005700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:30.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:30 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:30.122+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:30.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:30.034+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:30.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:30.034+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:30.654 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:30 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:30.190+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:30.654 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:30 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:30.190+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:30.912 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:30 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:30.655+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:30.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:30.694+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:31.139 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:30 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:30.691+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:31.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:31 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:31.140+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:31.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:31.023+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:31.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:31.062+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:31.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:31.062+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:31.661 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:31 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:31.196+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:31.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:31 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:31.196+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:31.912 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:31 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:31.662+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:31.926 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:31 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:31.666+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:31.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:31.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:31.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:31.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:32.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:32 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:32.161+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:32.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:32 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:32.161+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:32.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:32 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:32.148+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:32.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:32 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:32.148+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:32.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:32.033+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:32.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:32.017+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:32.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:32.017+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:32.627+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:32.627+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:33.127 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:32 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:32.689+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:33.127 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:32 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:32.689+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:33.147 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:32 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:32.760+0000 7fda7f5f9700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.101869+0000 front 2024-05-09T08:57:08.101893+0000 (oldest deadline 2024-05-09T08:57:32.202024+0000) 2024-05-09T08:57:33.148 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:32 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:32.687+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:33.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:33 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:33.148+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:33.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:33 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:33.148+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:33.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:33 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:33.128+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:33.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:33 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:33.128+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:33.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:33.053+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:33.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:33.027+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:33.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:33.027+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:33.912 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:33 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:33.737+0000 7fda7f5f9700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.101869+0000 front 2024-05-09T08:57:08.101893+0000 (oldest deadline 2024-05-09T08:57:32.202024+0000) 2024-05-09T08:57:33.912 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:33 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:33.647+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:33.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:33.629+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:33.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:33.629+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:34.115 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:33 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:33.712+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:34.115 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:33 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:33.712+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:34.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:34 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:34.115+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:34.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:34 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:34.115+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:34.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:34 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:34.079+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:34.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:34 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:34.031+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:34.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:34 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:34.031+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:34.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:34 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:34.170+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:34.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:34 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:34.170+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:34.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:34 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:34.638+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:34.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:34 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:34.638+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:35.106 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:34 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:34.737+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:35.166 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:34 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:34.737+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:35.167 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:34 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4[45474]: 2024-05-09T08:57:34.731+0000 7fda7f5f9700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.101869+0000 front 2024-05-09T08:57:08.101893+0000 (oldest deadline 2024-05-09T08:57:32.202024+0000) 2024-05-09T08:57:35.167 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:34 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:34.663+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:35.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:35 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:35.148+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:35.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:35 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:35.148+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:35.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:35 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:35.106+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:35.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:35 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:35.106+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:35.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:35 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:35.074+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:35.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:35 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:35.038+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:35.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:35 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:35.038+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:35.794 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:35 smithi160 podman[75261]: 2024-05-09 08:57:35.46382938 +0000 UTC m=+5.716466369 container died 87e4c25e21cd28efa7253d7c3fecff4b5908302d7e3b5c8df9acaa24aab15f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T08:57:35.795 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:35 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:35.690+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:35.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:35 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:35.596+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:35.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:35 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:35.596+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:36.061 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:35 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:35.772+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:36.061 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:35 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:35.772+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:36.095 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:35 smithi160 podman[75261]: 2024-05-09 08:57:35.886789827 +0000 UTC m=+6.139426813 container cleanup 87e4c25e21cd28efa7253d7c3fecff4b5908302d7e3b5c8df9acaa24aab15f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T08:57:36.095 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:35 smithi160 bash[75261]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4 2024-05-09T08:57:36.412 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75281]: 2024-05-09 08:57:36.096313668 +0000 UTC m=+0.631749146 container remove 87e4c25e21cd28efa7253d7c3fecff4b5908302d7e3b5c8df9acaa24aab15f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T08:57:36.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:36 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:36.099+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:36.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:36 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:36.099+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:36.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:36 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:36.060+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:36.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:36 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:36.060+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:36.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:36 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:36.113+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:36.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:36 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:36.011+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:36.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:36 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:36.011+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:36.865 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:36 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:36.669+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:36.866 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75374]: 2024-05-09 08:57:36.415600356 +0000 UTC m=+0.018250846 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:36.866 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75374]: 2024-05-09 08:57:36.566330359 +0000 UTC m=+0.168980849 container create 36ac6dd0eab484da08b2c561161acdfbaaaf862c0b904973aaeeaef6ffb08186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T08:57:36.866 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75374]: 2024-05-09 08:57:36.867324926 +0000 UTC m=+0.469975416 container init 36ac6dd0eab484da08b2c561161acdfbaaaf862c0b904973aaeeaef6ffb08186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:36.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:36 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:36.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:36.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:36 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:36.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:37.015 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:36 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:36.762+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:37.015 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:36 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:36.762+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:37.156 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:37 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:37.066+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:37.156 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:37 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:37.066+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:37.156 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75374]: 2024-05-09 08:57:36.871058959 +0000 UTC m=+0.473709457 container start 36ac6dd0eab484da08b2c561161acdfbaaaf862c0b904973aaeeaef6ffb08186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T08:57:37.156 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:36 smithi160 podman[75374]: 2024-05-09 08:57:36.958898561 +0000 UTC m=+0.561549065 container attach 36ac6dd0eab484da08b2c561161acdfbaaaf862c0b904973aaeeaef6ffb08186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T08:57:37.412 INFO:journalctl@ceph.osd.4.smithi160.stdout:May 09 08:57:37 smithi160 podman[75374]: 2024-05-09 08:57:37.157941165 +0000 UTC m=+0.760591664 container died 36ac6dd0eab484da08b2c561161acdfbaaaf862c0b904973aaeeaef6ffb08186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:37.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:37.016+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:37.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:37.016+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:37.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:37.083+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:37.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:37.022+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:37.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:37.022+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:37.769 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:37 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:37.672+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:37.789 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.4.service' 2024-05-09T08:57:37.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:37.868 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T08:57:37.868 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T08:57:37.868 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.5 2024-05-09T08:57:37.894 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:37.615+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:37.894 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:37.615+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:38.061 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:37.751+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:38.061 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:37.751+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:38.174 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:38.055+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:38.175 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:37 smithi188 systemd[1]: Stopping Ceph osd.5 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:38.175 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:37.994+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:38.175 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:37 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:37.994+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:38.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:38 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:38.044+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:38.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:38 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:38.044+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:38.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:38 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:38.044+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:38.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:38 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:38.061+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:38.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:38 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:38.061+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:38.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:38.175+0000 7f1253178700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:38.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:38.175+0000 7f1253178700 -1 osd.5 91 *** Got signal Terminated *** 2024-05-09T08:57:38.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:38.175+0000 7f1253178700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:38.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:38.661+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:38.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:38 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:38.661+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:39.061 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:38 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:38.687+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:39.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:38 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:38.793+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:39.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:38 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:38.793+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:39.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:38 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:38.793+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:39.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:39.028+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:39.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:39.028+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:39.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:39.028+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:39.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:39.062+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:39.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:39.062+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:39.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:39.062+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:39.434 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:39.027+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:39.434 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:39.027+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:39.434 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:39.098+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:39.434 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:39.098+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:39.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:39.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:39.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:39.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:39.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:39.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:40.074 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:39 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:39.731+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:40.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:39.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:40.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:39.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:40.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:39 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:39.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:40.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:40.059+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:40.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:40.059+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:40.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:40.059+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:40.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:40 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:40.075+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:40.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:40 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:40.075+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:40.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:40 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:40.075+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:40.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:39.992+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:40.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:39 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:39.992+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:40.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:40 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:40.095+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:40.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:40 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:40.095+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:40.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:40 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:40.632+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:40.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:40 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:40.632+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:40.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:40 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:40.632+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:41.033 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:40 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:40.759+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:41.033 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:40 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:40.759+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:41.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:40.856+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:41.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:40.856+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:41.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:40 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:40.856+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:41.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:41.080+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:41.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:41.080+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:41.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:41.080+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:41.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:41 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:41.034+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:41.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:41 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:41.034+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:41.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:41 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:41.034+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:41.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:41.050+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:41.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:41.050+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:41.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:41.020+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:41.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:41.020+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:41.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:41.020+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.346935+0000 front 2024-05-09T08:57:15.346976+0000 (oldest deadline 2024-05-09T08:57:40.047069+0000) 2024-05-09T08:57:41.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:41.662+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:41.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:41.662+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:41.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:41 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:41.662+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:42.082 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:41 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:41.798+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:42.082 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:41 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:41.798+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:42.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:41.886+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:42.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:41.886+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:42.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:41.886+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:42.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:42.100+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:42.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:42.100+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:42.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:42.100+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:42.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:42 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:42.083+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:42.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:42 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:42.083+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:42.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:42 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:42.083+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:42.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:42.063+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:42.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:42.063+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:42.432 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:42.048+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:42.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:42.048+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:42.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:42.048+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.346935+0000 front 2024-05-09T08:57:15.346976+0000 (oldest deadline 2024-05-09T08:57:40.047069+0000) 2024-05-09T08:57:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:42.689+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:42.689+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:43.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:42 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:42.689+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:43.117 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:42 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:42.817+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:43.117 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:42 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:42.817+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:43.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:42.872+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:43.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:42.872+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:43.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:42 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:42.872+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:43.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:43.052+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:43.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:43.052+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:43.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:43.052+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:43.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:43 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:43.118+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:43.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:43 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:43.118+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:43.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:43 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:43.118+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:43.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:43.020+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:43.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:43.020+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:43.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:43.073+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:55.042624+0000 front 2024-05-09T08:56:55.042634+0000 (oldest deadline 2024-05-09T08:57:22.643193+0000) 2024-05-09T08:57:43.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:43.073+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:04.344391+0000 front 2024-05-09T08:57:04.344503+0000 (oldest deadline 2024-05-09T08:57:29.544942+0000) 2024-05-09T08:57:43.433 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5[46622]: 2024-05-09T08:57:43.073+0000 7f124b764700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.346935+0000 front 2024-05-09T08:57:15.346976+0000 (oldest deadline 2024-05-09T08:57:40.047069+0000) 2024-05-09T08:57:43.932 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:43 smithi188 podman[75784]: 2024-05-09 08:57:43.683147003 +0000 UTC m=+5.617931176 container died ef7b771206a510933f798a831ceb32fae63d0db023b3d322d03e879da7ab7260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-05-09T08:57:43.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:43.642+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:43.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:43.642+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:43.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:43 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:43.642+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:44.144 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:43 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:43.773+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:44.144 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:43 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:43.773+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:44.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:43.884+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:44.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:43.884+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:44.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:43 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:43.884+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:44.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:44.090+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:44.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:44.090+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:44.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:44.090+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:44.359 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:44 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:44.057+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:44.359 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:44 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:44.057+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:44.360 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:44 smithi188 podman[75784]: 2024-05-09 08:57:44.050633663 +0000 UTC m=+5.985417834 container cleanup ef7b771206a510933f798a831ceb32fae63d0db023b3d322d03e879da7ab7260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc) 2024-05-09T08:57:44.360 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:44 smithi188 bash[75784]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5 2024-05-09T08:57:44.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:44 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:44.145+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:44.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:44 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:44.145+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:44.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:44 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:44.145+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:44.614 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:44 smithi188 podman[75802]: 2024-05-09 08:57:44.360631454 +0000 UTC m=+0.672726953 container remove ef7b771206a510933f798a831ceb32fae63d0db023b3d322d03e879da7ab7260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T08:57:44.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:44 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:44.614+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:44.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:44 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:44.614+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:44.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:44 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:44.614+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:44.877 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:44 smithi188 podman[75895]: 2024-05-09 08:57:44.614486546 +0000 UTC m=+0.017844852 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:44.877 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:44 smithi188 podman[75895]: 2024-05-09 08:57:44.761625569 +0000 UTC m=+0.164983863 container create af2e90073df3649048bd8b0c38f4bf0bd4d4035d0330517fa07badb5e1acf3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T08:57:45.125 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:44 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:44.816+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:45.125 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:44 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:44.816+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:44.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:44.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:44 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:44.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:45.097+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:45.097+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:45.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:45.097+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:45.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:45 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:45.081+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:45.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:45 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:45.082+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:45.182 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:45 smithi188 podman[75895]: 2024-05-09 08:57:45.01417997 +0000 UTC m=+0.417538262 container init af2e90073df3649048bd8b0c38f4bf0bd4d4035d0330517fa07badb5e1acf3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc) 2024-05-09T08:57:45.182 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:45 smithi188 podman[75895]: 2024-05-09 08:57:45.017235651 +0000 UTC m=+0.420593951 container start af2e90073df3649048bd8b0c38f4bf0bd4d4035d0330517fa07badb5e1acf3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T08:57:45.182 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:45 smithi188 podman[75895]: 2024-05-09 08:57:45.089636314 +0000 UTC m=+0.492994626 container attach af2e90073df3649048bd8b0c38f4bf0bd4d4035d0330517fa07badb5e1acf3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T08:57:45.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:45 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:45.126+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:45.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:45 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:45.126+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:45.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:45 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:45.126+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:45.616 INFO:journalctl@ceph.osd.5.smithi188.stdout:May 09 08:57:45 smithi188 podman[75895]: 2024-05-09 08:57:45.305117764 +0000 UTC m=+0.708476048 container died af2e90073df3649048bd8b0c38f4bf0bd4d4035d0330517fa07badb5e1acf3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T08:57:45.900 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:45 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:45.639+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:45.901 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:45 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:45.639+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:45.901 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:45 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:45.639+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:45.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.5.service' 2024-05-09T08:57:46.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:46.008 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T08:57:46.008 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T08:57:46.009 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.6 2024-05-09T08:57:46.111 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:45 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:45.831+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:46.111 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:45 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:45.831+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:46.135 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:45.880+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:46.135 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:45.880+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:46.135 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:45.880+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:46.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:46 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:46.081+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:46.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:46 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:46.081+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:46.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:46.112+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:46.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:46.112+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:46.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:46.112+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:46.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 systemd[1]: Stopping Ceph osd.6 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:46.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.144+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:46.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.144+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:46.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.144+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:46.924 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.600+0000 7fdb8a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:46.925 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.600+0000 7fdb8a600700 -1 osd.6 91 *** Got signal Terminated *** 2024-05-09T08:57:46.925 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:46.600+0000 7fdb8a600700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:46.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:46 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:46.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:46.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:46 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:46.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:46.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:46 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:46.672+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:47.149 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:46.862+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:47.149 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:46 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:46.862+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:47.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:46.925+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:47.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:46.925+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:47.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:46 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:46.925+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:47.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:47.125+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:47.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:47.125+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:47.178 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:47.125+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:47.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:47 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:47.149+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:47.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:47 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:47.149+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:47.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:47 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:47.149+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:47.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:47 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:47.131+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:47.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:47 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:47.131+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:48.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:47 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:47.906+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:49.168 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:47 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:47.906+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:49.169 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:48.111+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:48.111+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:48.111+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:47 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:47.714+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:47 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:47.714+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:47 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:47.714+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:49.170 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:48.166+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:48.166+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:48.166+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:48.166+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:48.165+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:48.165+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:48.165+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:48.873+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:48.873+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:49.171 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:48 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:48.873+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:47.881+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:47.881+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:47.881+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:47 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:47.881+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:48.857+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:48.857+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:48.857+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:49.172 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:48 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:48.857+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:49.173 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:48.715+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:49.173 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:48.715+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:49.173 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:48 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:48.715+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:49.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:49.138+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:49.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:49.138+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:49.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:49.138+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:49.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:49.138+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:49.430 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:49.146+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:49.430 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:49.146+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:49.430 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:49.146+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:49.433 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:49.142+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:49.433 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:49.142+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:49.433 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:49.142+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:49.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:49.688+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:49.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:49.688+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:49.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:49.688+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:49.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:49 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:49.688+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:50.130 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:49.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:50.130 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:49.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:50.130 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:49.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:50.130 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:49.878+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:50.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:49.900+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:50.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:49.900+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:50.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:49 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:49.900+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:50.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:50.134+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:50.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:50.134+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:50.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:50.134+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:50.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:50.134+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:50.367 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:50.094+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:50.367 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:50.094+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:50.367 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:50.094+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:50.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:50.131+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:50.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:50.131+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:50.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:50.131+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:51.123 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:50.736+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:51.123 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:50.736+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:51.123 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:50.736+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:51.123 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:50 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:50.736+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:51.151 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:50.883+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:51.151 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:50.883+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:51.151 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:50.883+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:51.151 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:50 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:50.883+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:51.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:50.923+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:51.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:50.923+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:51.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:50 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:50.923+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:51.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:51.090+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:51.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:51.090+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:51.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:51.090+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:51.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:51.090+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:51.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:51.151+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.093698+0000 front 2024-05-09T08:56:57.093846+0000 (oldest deadline 2024-05-09T08:57:24.096030+0000) 2024-05-09T08:57:51.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:51.151+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:06.399539+0000 front 2024-05-09T08:57:06.399316+0000 (oldest deadline 2024-05-09T08:57:31.699314+0000) 2024-05-09T08:57:51.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6[57765]: 2024-05-09T08:57:51.151+0000 7fdb85e00700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.601332+0000 front 2024-05-09T08:57:14.601699+0000 (oldest deadline 2024-05-09T08:57:38.101572+0000) 2024-05-09T08:57:51.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:51.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:51.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:51.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:51.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:51.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:52.134 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:51.699+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:52.134 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:51.699+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:52.134 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:51.699+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:52.135 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:51 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:51.699+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:52.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:51.884+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:52.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:51.884+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:52.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:51 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:51.884+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:52.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:52.046+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:52.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:52.046+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:52.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:52.046+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:52.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:52.046+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:52.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:51.904+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:52.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:51.904+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:52.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:51.904+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:52.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:51 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:51.904+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:52.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:52.134+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:52.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:52.135+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:52.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:52.135+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:52.473 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:52 smithi133 podman[90919]: 2024-05-09 08:57:52.197828485 +0000 UTC m=+5.744810871 container died 265ea888be613b038fcc80279ca0c1ad9a5a3c58b356b2aacbb769e83b1b212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T08:57:52.869 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:52 smithi133 podman[90919]: 2024-05-09 08:57:52.610633181 +0000 UTC m=+6.157615560 container cleanup 265ea888be613b038fcc80279ca0c1ad9a5a3c58b356b2aacbb769e83b1b212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-09T08:57:52.869 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:52 smithi133 bash[90919]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6 2024-05-09T08:57:52.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:52.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:52.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:52.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:52.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:52.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:52.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:52 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:52.664+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:53.177 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:52 smithi133 podman[90939]: 2024-05-09 08:57:52.886747083 +0000 UTC m=+0.676005249 container remove 265ea888be613b038fcc80279ca0c1ad9a5a3c58b356b2aacbb769e83b1b212a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:57:53.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:52 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:52.868+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:53.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:52 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:52.868+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:53.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:52 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:52.868+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:53.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:52 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:52.868+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.002+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.002+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.002+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.002+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:52.925+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:53.370 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:52.925+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:53.371 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:52 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:52.925+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:53.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:53.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:53.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:53.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:53.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:53.123+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:53.835+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:53.835+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:53.835+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:53 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:53.835+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:53 smithi133 podman[91029]: 2024-05-09 08:57:53.595078655 +0000 UTC m=+0.036667248 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:57:53.985 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:53 smithi133 podman[91029]: 2024-05-09 08:57:53.721077825 +0000 UTC m=+0.162666438 container create a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:54.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:53.695+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:54.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:53.695+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:54.170 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:53.695+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:54.171 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:53 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:53.695+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:54.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:53.935+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:54.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:53.935+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:54.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:53.935+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:54.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.975+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:54.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.975+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:54.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.975+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:54.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:53.975+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:54.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:53 smithi133 podman[91029]: 2024-05-09 08:57:53.985816973 +0000 UTC m=+0.427405592 container init a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc) 2024-05-09T08:57:54.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:53 smithi133 podman[91029]: 2024-05-09 08:57:53.997085892 +0000 UTC m=+0.438674512 container start a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-09T08:57:54.427 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:54 smithi133 podman[91029]: 2024-05-09 08:57:54.098467287 +0000 UTC m=+0.540055963 container attach a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:54.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:54.171+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:54.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:54.171+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:54.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:54.171+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:54.788 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:54 smithi133 podman[91029]: 2024-05-09 08:57:54.454728001 +0000 UTC m=+0.896316584 container died a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-96805dc) 2024-05-09T08:57:54.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:54.651+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:54.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:54.651+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:54.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:54.651+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:54.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:54 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:54.651+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:55.116 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:54.787+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:55.116 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:54.787+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:55.116 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:54.787+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:55.117 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:54 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:54.787+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:55.305 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.6.service' 2024-05-09T08:57:55.373 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:55 smithi133 podman[91057]: 2024-05-09 08:57:55.26946136 +0000 UTC m=+0.799682031 container remove a72de667a9745c55757b03eea49bf1363ba913ab3a10538dc6fe67a77380cce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T08:57:55.373 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:55 smithi133 systemd[1]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.6.service: Deactivated successfully. 2024-05-09T08:57:55.373 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:55 smithi133 systemd[1]: Stopped Ceph osd.6 for 99f031e6-0ddc-11ef-bc98-c7b262605968. 2024-05-09T08:57:55.373 INFO:journalctl@ceph.osd.6.smithi133.stdout:May 09 08:57:55 smithi133 systemd[1]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.6.service: Consumed 8min 32.859s CPU time. 2024-05-09T08:57:55.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:57:55.404 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T08:57:55.405 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T08:57:55.405 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.7 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:54.956+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:54.956+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:54.956+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:54.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:54.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:54.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:55.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:54 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:54.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:55.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:55.147+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:55.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:55.147+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:55.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:55.147+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:55.723 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 systemd[1]: Stopping Ceph osd.7 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:57:56.045 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:55.781+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:56.046 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:55.781+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:56.046 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:55.781+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:56.046 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:55 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:55.781+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:56.111 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:55.686+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:56.112 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:55.686+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:56.112 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:55.686+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:56.112 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:55 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:55.686+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:56.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:55.934+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:56.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:55.934+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:56.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:55.934+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:55.934+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.723+0000 7f452a844700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.723+0000 7f452a844700 -1 osd.7 91 *** Got signal Terminated *** 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.723+0000 7f452a844700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:56.164 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:55 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:55.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.603812+0000 front 2024-05-09T08:57:34.603834+0000 (oldest deadline 2024-05-09T08:57:55.703950+0000) 2024-05-09T08:57:56.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:56.112+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:56.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:56.112+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:56.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:56.112+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:56.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:56.653+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:56.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:56.653+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:56.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:56.653+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:56.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:56 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:56.653+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:57.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:56.990+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:57.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:56.990+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:57.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:56.990+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:57.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:56.990+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.603812+0000 front 2024-05-09T08:57:34.603834+0000 (oldest deadline 2024-05-09T08:57:55.703950+0000) 2024-05-09T08:57:57.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:56.911+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:57.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:56.911+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:57.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:56.911+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:57.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:56.911+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:57.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:56.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:57.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:56.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:57.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:56.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:57.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:56.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:57.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:56 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:56.829+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:57:57.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:57.155+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:57.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:57.155+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:57.433 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:57.155+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:57.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:57.616+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:57.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:57.616+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:57.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:57.616+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:57.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:57.616+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:57.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:57 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:57.616+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:57:58.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:57 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:57.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:58.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:57 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:57.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:58.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:57 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:57.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:58.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:57 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:57.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:58.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:57 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:57.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:57:58.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.009+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:58.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.009+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:58.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.009+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:58.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.009+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.603812+0000 front 2024-05-09T08:57:34.603834+0000 (oldest deadline 2024-05-09T08:57:55.703950+0000) 2024-05-09T08:57:58.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:57 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:57.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:58.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:57 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:57.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:58.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:57 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:57.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:58.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:57 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:57.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:58.581 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:58.188+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:58.581 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:58.188+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:58.581 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:58.188+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:58.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:58.581+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:58.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:58.581+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:58.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:58.581+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:58.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:58.581+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:58.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:58 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:58.581+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:57:59.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:58 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:58.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:57:59.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:58 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:58.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:57:59.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:58 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:58.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:57:59.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:58 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:58.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:57:59.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:58 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:58.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:57:59.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:57:59.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:57:59.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:57:59.413 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:58.968+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.603812+0000 front 2024-05-09T08:57:34.603834+0000 (oldest deadline 2024-05-09T08:57:55.703950+0000) 2024-05-09T08:57:59.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:58.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:57:59.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:58.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:57:59.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:58.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:57:59.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:58 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:58.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:57:59.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:59.180+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:57:59.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:59.180+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:57:59.432 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:57:59.180+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:57:59.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:59.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:57:59.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:59.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:57:59.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:59.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:57:59.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:59.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:57:59.877 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:57:59 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:57:59.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:00.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:59.826+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:00.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:59.826+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:00.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:59.826+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:00.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:59.826+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:00.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:57:59 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:57:59.826+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:59.937+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.498383+0000 front 2024-05-09T08:57:08.498437+0000 (oldest deadline 2024-05-09T08:57:29.598529+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:59.937+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:16.600147+0000 front 2024-05-09T08:57:16.600177+0000 (oldest deadline 2024-05-09T08:57:40.100113+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:59.937+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.602829+0000 front 2024-05-09T08:57:24.602773+0000 (oldest deadline 2024-05-09T08:57:48.702714+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7[52018]: 2024-05-09T08:57:59.937+0000 7f4522e30700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.603812+0000 front 2024-05-09T08:57:34.603834+0000 (oldest deadline 2024-05-09T08:57:55.703950+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:59.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:59.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:00.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:59.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:00.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:59.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:00.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:57:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:57:59.926+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:00.589 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:00.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:00.589 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:00.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:00.589 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:00.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:00.589 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:00.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:00.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:00.589+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:00.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:00.589+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:00.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:00.589+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:00.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:00.589+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:00.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:00 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:00.589+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:01.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:00 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:00.779+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:01.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:00 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:00.779+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:01.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:00 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:00.779+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:01.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:00 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:00.779+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:01.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:00 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:00.779+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:01.309 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:00 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:00.961+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:01.309 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:00 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:00.961+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:01.309 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:00 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:00.961+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:01.309 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:00 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:00.961+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:01.309 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:00 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:00.961+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:01.564 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:01.265+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:01.564 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:01.265+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:01.564 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:01.265+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:01.564 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:01.265+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:01.589 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:01 smithi160 podman[75503]: 2024-05-09 08:58:01.310339062 +0000 UTC m=+5.711394525 container died 12f304b8c9f5930a0f92058c89b1886b3f11c954a7603838b31f5ee0497b8810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-09T08:58:01.848 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:01 smithi160 podman[75503]: 2024-05-09 08:58:01.657150205 +0000 UTC m=+6.058205670 container cleanup 12f304b8c9f5930a0f92058c89b1886b3f11c954a7603838b31f5ee0497b8810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T08:58:01.848 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:01 smithi160 bash[75503]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7 2024-05-09T08:58:01.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:01.564+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:01.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:01.564+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:01.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:01.564+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:01.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:01.564+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:01.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:01 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:01.564+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:01 smithi160 podman[75520]: 2024-05-09 08:58:01.850141987 +0000 UTC m=+0.534584299 container remove 12f304b8c9f5930a0f92058c89b1886b3f11c954a7603838b31f5ee0497b8810 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:01 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:01.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:01 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:01.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:01 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:01.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:01 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:01.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:02.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:01 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:01.935+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:02.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:01.746+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:02.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:01.746+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:02.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:01.746+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:02.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:01.746+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:02.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:01 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:01.746+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:02.535 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.164961571 +0000 UTC m=+0.018347144 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:58:02.535 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.2845483 +0000 UTC m=+0.137933899 container create 4300c4994cc2644df8233da45cdf7c37159dce3af450aa589530fe7ab9f0e78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T08:58:02.567 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:02.285+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:02.567 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:02.285+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:02.567 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:02.285+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:02.567 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:02.285+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:02.829 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.537108434 +0000 UTC m=+0.390494009 container init 4300c4994cc2644df8233da45cdf7c37159dce3af450aa589530fe7ab9f0e78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T08:58:02.829 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.540528191 +0000 UTC m=+0.393913755 container start 4300c4994cc2644df8233da45cdf7c37159dce3af450aa589530fe7ab9f0e78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:58:02.829 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.637056752 +0000 UTC m=+0.490442324 container attach 4300c4994cc2644df8233da45cdf7c37159dce3af450aa589530fe7ab9f0e78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T08:58:02.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:02.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:02.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:02.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:02.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:02.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:02.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:02.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:02.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:02 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:02.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.7.smithi160.stdout:May 09 08:58:02 smithi160 podman[75613]: 2024-05-09 08:58:02.830975441 +0000 UTC m=+0.684360994 container died 4300c4994cc2644df8233da45cdf7c37159dce3af450aa589530fe7ab9f0e78d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:02 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:02.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:02 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:02.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:02 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:02.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:02 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:02.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:03.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:02 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:02.957+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:03.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:02.739+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:03.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:02.739+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:03.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:02.739+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:03.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:02.739+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:03.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:02 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:02.739+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:03.513 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.7.service' 2024-05-09T08:58:03.551 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.235+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:03.552 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.235+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:03.552 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.235+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:03.552 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.235+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:03.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:03.588 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T08:58:03.589 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T08:58:03.589 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.8 2024-05-09T08:58:03.918 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 systemd[1]: Stopping Ceph osd.8 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:03.919 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:03.552+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:03.919 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:03.552+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:03.919 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:03.552+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:03.919 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:03.552+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:03.919 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:03.552+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:04.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:03.777+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:04.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:03.777+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:04.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:03.777+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:04.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:03.777+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:04.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:03 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:03.777+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:04.181 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.919+0000 7faff45a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:04.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.919+0000 7faff45a1700 -1 osd.8 91 *** Got signal Terminated *** 2024-05-09T08:58:04.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:03 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:03.919+0000 7faff45a1700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:58:04.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:03.945+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:04.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:03.945+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:04.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:03.945+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:04.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:03.945+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:04.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:03 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:03.945+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:04.582 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:04.231+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:04.582 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:04.231+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:04.582 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:04.231+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:04.582 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:04.231+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:04.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:04.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:04.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:04.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:04.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:04.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:04.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:04.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:04.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:04 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:04.583+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:05.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:04.809+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:05.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:04.809+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:05.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:04.809+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:05.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:04.809+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:05.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:04 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:04.809+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:05.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:04.965+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:05.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:04.965+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:05.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:04.965+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:05.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:04.965+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:05.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:04.965+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:05.595 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:05.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:05.595 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:05.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:05.595 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:05.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:05.595 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:05.227+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:05.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:05.595+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:05.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:05.595+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:05.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:05.595+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:05.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:05.595+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:05.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:05 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:05.595+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:06.144 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:05 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:05.837+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:06.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:05 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:05.936+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:06.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:05 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:05.936+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:06.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:05 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:05.936+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:06.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:05 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:05.936+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:06.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:05 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:05.936+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:06.579 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:06.268+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:06.580 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:06.268+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:06.580 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:06.268+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:06.580 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:06.268+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:06.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:06 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:06.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:07.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:07.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:07.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:07.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:07.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:07.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:06 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:06.849+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:07.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:06 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:06.915+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:07.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:06 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:06.915+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:07.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:06 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:06.915+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:07.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:06 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:06.915+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:07.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:06 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:06.915+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:07.567 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:07.308+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:07.568 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:07.308+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:07.568 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:07.308+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:07.568 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:07.308+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:07.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:07.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:07.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:07.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:07.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:07.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:07 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:07.568+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:08.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:08.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:08.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:08.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:08.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:08.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:07 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:07.814+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:08.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:07 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:07.922+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:08.538 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:08.269+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:05.369254+0000 front 2024-05-09T08:57:05.369152+0000 (oldest deadline 2024-05-09T08:57:30.069081+0000) 2024-05-09T08:58:08.538 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:08.269+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.070802+0000 front 2024-05-09T08:57:14.070775+0000 (oldest deadline 2024-05-09T08:57:38.771033+0000) 2024-05-09T08:58:08.538 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:08.269+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.573023+0000 front 2024-05-09T08:57:24.573071+0000 (oldest deadline 2024-05-09T08:57:48.073051+0000) 2024-05-09T08:58:08.538 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:08.269+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:33.974399+0000 front 2024-05-09T08:57:33.974345+0000 (oldest deadline 2024-05-09T08:57:59.274343+0000) 2024-05-09T08:58:08.538 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8[53183]: 2024-05-09T08:58:08.269+0000 7fafecb8d700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.575408+0000 front 2024-05-09T08:57:41.575479+0000 (oldest deadline 2024-05-09T08:58:07.376121+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:08.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:08.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:09.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:09.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:09.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:09.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:09.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:09.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:08 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:08.875+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:09.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:09.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:09.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:09.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:09.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:09.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:08 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:08.848+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:09.871 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:09 smithi188 podman[76013]: 2024-05-09 08:58:09.584147113 +0000 UTC m=+5.797347206 container died c5726f70ba8529eebb49620d60c6d5333a083192ebef0ceaeda96e711f3c46f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:09.872 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:09 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:09.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:10.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:09 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:09.863+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:10.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:09 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:09.890+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:10.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:09 smithi188 podman[76013]: 2024-05-09 08:58:09.960789559 +0000 UTC m=+6.173989650 container cleanup c5726f70ba8529eebb49620d60c6d5333a083192ebef0ceaeda96e711f3c46f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-09T08:58:10.182 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:09 smithi188 bash[76013]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8 2024-05-09T08:58:10.613 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:10 smithi188 podman[76030]: 2024-05-09 08:58:10.270650942 +0000 UTC m=+0.682521069 container remove c5726f70ba8529eebb49620d60c6d5333a083192ebef0ceaeda96e711f3c46f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T08:58:10.932 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:10 smithi188 podman[76121]: 2024-05-09 08:58:10.527372395 +0000 UTC m=+0.018358450 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:58:10.932 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:10 smithi188 podman[76121]: 2024-05-09 08:58:10.629868376 +0000 UTC m=+0.120854477 container create 2cd019c4c8725dcd10ef57ba475dadb7cf824abd3292dc16ab354ad1a19d9b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T08:58:10.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:10.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:10.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:10.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:10.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:10.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:10 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:10.613+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:11.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:10 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:10.882+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:11.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:10 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:10.900+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:11.264 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:10 smithi188 podman[76121]: 2024-05-09 08:58:10.965830594 +0000 UTC m=+0.456816651 container init 2cd019c4c8725dcd10ef57ba475dadb7cf824abd3292dc16ab354ad1a19d9b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-09T08:58:11.264 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:10 smithi188 podman[76121]: 2024-05-09 08:58:10.968969307 +0000 UTC m=+0.459955367 container start 2cd019c4c8725dcd10ef57ba475dadb7cf824abd3292dc16ab354ad1a19d9b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-96805dc, GIT_BRANCH=HEAD) 2024-05-09T08:58:11.264 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:11 smithi188 podman[76121]: 2024-05-09 08:58:11.057490094 +0000 UTC m=+0.548476151 container attach 2cd019c4c8725dcd10ef57ba475dadb7cf824abd3292dc16ab354ad1a19d9b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, io.buildah.version=1.33.5) 2024-05-09T08:58:11.535 INFO:journalctl@ceph.osd.8.smithi188.stdout:May 09 08:58:11 smithi188 podman[76121]: 2024-05-09 08:58:11.266488455 +0000 UTC m=+0.757474520 container died 2cd019c4c8725dcd10ef57ba475dadb7cf824abd3292dc16ab354ad1a19d9b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-8-deactivate, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:11.852 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:11 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:11.580+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:11.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.8.service' 2024-05-09T08:58:11.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:11.908 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T08:58:11.909 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T08:58:11.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.9 2024-05-09T08:58:12.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:12.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:12.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:12.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:12.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:12.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:11 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:11.875+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:12.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 systemd[1]: Stopping Ceph osd.9 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:12.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:12.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:12.412 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:12.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:12.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:12.413 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:11 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:11.921+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:12.831 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.444+0000 7fa09e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:12.831 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.444+0000 7fa09e400700 -1 osd.9 91 *** Got signal Terminated *** 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.444+0000 7fa09e400700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:12.832 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:12 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:12.831+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:12.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:12.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:12.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:12.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:12.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:12.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:12.602+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:13.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:13.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:13.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:13.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:13.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:13.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:12 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:12.874+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:13.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:13.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:13.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:13.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:13.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:13.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:13 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:13.557+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:14.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:13 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:13.880+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:14.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:14.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:14.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:14.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:14.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:14.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:13 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:13.823+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:14.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:14 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:14.549+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:15.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:14 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:14.832+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:15.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:15.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:15.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:15.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:15.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:15.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:15.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:14 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:14.818+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:51.192981+0000 front 2024-05-09T08:57:51.193134+0000 (oldest deadline 2024-05-09T08:58:13.994165+0000) 2024-05-09T08:58:15.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:15.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:15.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:15.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:15.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:15.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:15 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:15.515+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:16.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:15 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:15.850+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:16.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:16.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:16.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:16.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:16.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:16.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:16.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:15 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:15.776+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:51.192981+0000 front 2024-05-09T08:57:51.193134+0000 (oldest deadline 2024-05-09T08:58:13.994165+0000) 2024-05-09T08:58:16.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:16.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:16.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:16.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:16.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:16.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:16 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:16.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:17.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:17.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:17.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:17.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:17.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:17.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:17.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:16 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:16.871+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.767910+0000 front 2024-05-09T08:56:57.768037+0000 (oldest deadline 2024-05-09T08:57:23.668130+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.971484+0000 front 2024-05-09T08:57:08.971552+0000 (oldest deadline 2024-05-09T08:57:32.373840+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:15.079857+0000 front 2024-05-09T08:57:15.079954+0000 (oldest deadline 2024-05-09T08:57:38.579664+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:25.582963+0000 front 2024-05-09T08:57:25.583133+0000 (oldest deadline 2024-05-09T08:57:47.283064+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:31.385458+0000 front 2024-05-09T08:57:31.385864+0000 (oldest deadline 2024-05-09T08:57:56.685460+0000) 2024-05-09T08:58:17.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.289420+0000 front 2024-05-09T08:57:41.289502+0000 (oldest deadline 2024-05-09T08:58:05.389645+0000) 2024-05-09T08:58:17.178 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:16 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9[64331]: 2024-05-09T08:58:16.790+0000 7fa099c00700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:51.192981+0000 front 2024-05-09T08:57:51.193134+0000 (oldest deadline 2024-05-09T08:58:13.994165+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:17.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:17.497+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:18.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:17 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:17.861+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:18.325 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:18 smithi133 podman[91149]: 2024-05-09 08:58:18.050412157 +0000 UTC m=+5.741815960 container died 595d91a1422a66a31365d54bfdd577b80d9014e954bf360a28cd7b1543336d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T08:58:18.628 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:18 smithi133 podman[91149]: 2024-05-09 08:58:18.436873834 +0000 UTC m=+6.128277630 container cleanup 595d91a1422a66a31365d54bfdd577b80d9014e954bf360a28cd7b1543336d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True) 2024-05-09T08:58:18.629 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:18 smithi133 bash[91149]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:18.833 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:18 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:18.505+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:18.926 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:18 smithi133 podman[91168]: 2024-05-09 08:58:18.629415709 +0000 UTC m=+0.566153169 container remove 595d91a1422a66a31365d54bfdd577b80d9014e954bf360a28cd7b1543336d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:19.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:19.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:19.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:19.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:19.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:19.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:19.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:18 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:18.829+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:19.844 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:19 smithi133 podman[91257]: 2024-05-09 08:58:19.375131396 +0000 UTC m=+0.042853111 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:58:19.845 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:19 smithi133 podman[91257]: 2024-05-09 08:58:19.530271834 +0000 UTC m=+0.197993578 container create 910c18ac7ec844260c6e602a030ccd344a5267bc621c99cd5a88ca1fac8b78f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T08:58:19.845 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:19 smithi133 podman[91257]: 2024-05-09 08:58:19.844886352 +0000 UTC m=+0.512608122 container init 910c18ac7ec844260c6e602a030ccd344a5267bc621c99cd5a88ca1fac8b78f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:19.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:19 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:19.494+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:20.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:19 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:19.806+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:20.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:19 smithi133 podman[91257]: 2024-05-09 08:58:19.855525265 +0000 UTC m=+0.523247012 container start 910c18ac7ec844260c6e602a030ccd344a5267bc621c99cd5a88ca1fac8b78f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:58:20.177 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:19 smithi133 podman[91257]: 2024-05-09 08:58:19.93281012 +0000 UTC m=+0.600531907 container attach 910c18ac7ec844260c6e602a030ccd344a5267bc621c99cd5a88ca1fac8b78f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T08:58:20.630 INFO:journalctl@ceph.osd.9.smithi133.stdout:May 09 08:58:20 smithi133 podman[91257]: 2024-05-09 08:58:20.298403528 +0000 UTC m=+0.966125304 container died 910c18ac7ec844260c6e602a030ccd344a5267bc621c99cd5a88ca1fac8b78f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-09T08:58:20.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:20.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:20.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:20.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:20.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:20.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:20.541+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:20.968 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.9.service' 2024-05-09T08:58:21.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:21.069 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T08:58:21.069 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T08:58:21.069 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.10 2024-05-09T08:58:21.115 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:21.116 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:20 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:20.852+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:21.382 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 systemd[1]: Stopping Ceph osd.10 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:21.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.383+0000 7f63be754700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:21.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.383+0000 7f63be754700 -1 osd.10 91 *** Got signal Terminated *** 2024-05-09T08:58:21.662 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.383+0000 7f63be754700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:58:21.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:21.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:21.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:21.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:21.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:21.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:21 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:21.504+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:22.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:22.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:22.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:22.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:22.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:22.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:22.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:21 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:21.890+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:22.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:22.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:22.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:22.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:22.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:22.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:22 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:22.509+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:23.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:22 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:22.862+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:23.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:23 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:23.493+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:24.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:23 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:23.841+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:24.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:24 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:24.520+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:25.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:25.163 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:24 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:24.812+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:25.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:25 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:25.567+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:59.369959+0000 front 2024-05-09T08:56:59.369941+0000 (oldest deadline 2024-05-09T08:57:20.469939+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:07.471744+0000 front 2024-05-09T08:57:07.471884+0000 (oldest deadline 2024-05-09T08:57:29.771957+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:14.473672+0000 front 2024-05-09T08:57:14.473583+0000 (oldest deadline 2024-05-09T08:57:37.973452+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:23.675664+0000 front 2024-05-09T08:57:23.675799+0000 (oldest deadline 2024-05-09T08:57:47.175439+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:34.077435+0000 front 2024-05-09T08:57:34.077482+0000 (oldest deadline 2024-05-09T08:57:59.377541+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:41.078477+0000 front 2024-05-09T08:57:41.078575+0000 (oldest deadline 2024-05-09T08:58:06.978476+0000) 2024-05-09T08:58:26.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:25 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10[58789]: 2024-05-09T08:58:25.773+0000 7f63bad48700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2024-05-09T08:57:50.479879+0000 front 2024-05-09T08:57:50.479782+0000 (oldest deadline 2024-05-09T08:58:13.379628+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:26.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:26 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:26.539+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:27.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:26 smithi160 podman[75718]: 2024-05-09 08:58:26.909259875 +0000 UTC m=+5.648444745 container died 79f4ca60ad6f89204b19da6be18ec752f5d34d57009472c3ba481fb1cd3efef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T08:58:27.559 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:27 smithi160 podman[75718]: 2024-05-09 08:58:27.267615578 +0000 UTC m=+6.006800493 container cleanup 79f4ca60ad6f89204b19da6be18ec752f5d34d57009472c3ba481fb1cd3efef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-09T08:58:27.559 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:27 smithi160 bash[75718]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10 2024-05-09T08:58:27.855 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:27 smithi160 podman[75747]: 2024-05-09 08:58:27.56070827 +0000 UTC m=+0.655002142 container remove 79f4ca60ad6f89204b19da6be18ec752f5d34d57009472c3ba481fb1cd3efef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-09T08:58:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:27.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:27.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:27.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:27 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:27.521+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:28.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:27 smithi160 podman[75842]: 2024-05-09 08:58:27.760247886 +0000 UTC m=+0.017908465 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:58:28.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:27 smithi160 podman[75842]: 2024-05-09 08:58:27.870064132 +0000 UTC m=+0.127724723 container create 18d04e8e33deb4f003e45d9e762b1df597c79edcfa8407fcf526cf836fb249ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T08:58:28.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:28 smithi160 podman[75842]: 2024-05-09 08:58:28.08122145 +0000 UTC m=+0.338882009 container init 18d04e8e33deb4f003e45d9e762b1df597c79edcfa8407fcf526cf836fb249ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:58:28.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:28 smithi160 podman[75842]: 2024-05-09 08:58:28.084574501 +0000 UTC m=+0.342235048 container start 18d04e8e33deb4f003e45d9e762b1df597c79edcfa8407fcf526cf836fb249ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-05-09T08:58:28.162 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:28 smithi160 podman[75842]: 2024-05-09 08:58:28.163985891 +0000 UTC m=+0.421646454 container attach 18d04e8e33deb4f003e45d9e762b1df597c79edcfa8407fcf526cf836fb249ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-09T08:58:28.607 INFO:journalctl@ceph.osd.10.smithi160.stdout:May 09 08:58:28 smithi160 podman[75842]: 2024-05-09 08:58:28.378644243 +0000 UTC m=+0.636304794 container died 18d04e8e33deb4f003e45d9e762b1df597c79edcfa8407fcf526cf836fb249ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:28.890 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:28 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:28.503+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:28.951 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.10.service' 2024-05-09T08:58:28.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:28.988 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T08:58:28.988 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T08:58:28.988 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.11 2024-05-09T08:58:29.182 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 systemd[1]: Stopping Ceph osd.11 for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.295+0000 7f4609f3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.295+0000 7f4609f3a700 -1 osd.11 91 *** Got signal Terminated *** 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.295+0000 7f4609f3a700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:29.545 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:29 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:29.546+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:30.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:30.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:30.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:30.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:30.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:30.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:30.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:30 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:30.562+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:31.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:31.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:31.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:31.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:31.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:31.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:31.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:31.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:31 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:31.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-05-09T08:58:05.997137+0000 front 2024-05-09T08:58:05.997115+0000 (oldest deadline 2024-05-09T08:58:30.597600+0000) 2024-05-09T08:58:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:32.932 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:32.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:32.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:32.933 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:32 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:32.599+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-05-09T08:58:05.997137+0000 front 2024-05-09T08:58:05.997115+0000 (oldest deadline 2024-05-09T08:58:30.597600+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2024-05-09T08:56:57.783446+0000 front 2024-05-09T08:56:57.783467+0000 (oldest deadline 2024-05-09T08:57:21.283465+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6804 osd.1 since back 2024-05-09T08:57:08.885089+0000 front 2024-05-09T08:57:08.884948+0000 (oldest deadline 2024-05-09T08:57:31.185122+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-05-09T08:57:17.086140+0000 front 2024-05-09T08:57:17.086172+0000 (oldest deadline 2024-05-09T08:57:38.786247+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2024-05-09T08:57:24.389037+0000 front 2024-05-09T08:57:24.388827+0000 (oldest deadline 2024-05-09T08:57:49.088822+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6812 osd.4 since back 2024-05-09T08:57:32.490443+0000 front 2024-05-09T08:57:32.490496+0000 (oldest deadline 2024-05-09T08:57:57.190425+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-05-09T08:57:42.892213+0000 front 2024-05-09T08:57:42.892196+0000 (oldest deadline 2024-05-09T08:58:06.392180+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.160:6820 osd.7 since back 2024-05-09T08:57:58.495457+0000 front 2024-05-09T08:57:58.495434+0000 (oldest deadline 2024-05-09T08:58:24.395477+0000) 2024-05-09T08:58:33.934 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:33 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11[60212]: 2024-05-09T08:58:33.645+0000 7f4602526700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-05-09T08:58:05.997137+0000 front 2024-05-09T08:58:05.997115+0000 (oldest deadline 2024-05-09T08:58:30.597600+0000) 2024-05-09T08:58:35.036 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:34 smithi188 podman[76238]: 2024-05-09 08:58:34.728515176 +0000 UTC m=+5.551944854 container died a67deaac42013f0081c9e4f729c1fce626116603b52411cc9b681306895b9f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T08:58:35.395 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76238]: 2024-05-09 08:58:35.111475547 +0000 UTC m=+5.934905235 container cleanup a67deaac42013f0081c9e4f729c1fce626116603b52411cc9b681306895b9f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T08:58:35.395 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 bash[76238]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11 2024-05-09T08:58:35.682 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76256]: 2024-05-09 08:58:35.396217228 +0000 UTC m=+0.658504678 container remove a67deaac42013f0081c9e4f729c1fce626116603b52411cc9b681306895b9f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-09T08:58:35.989 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76346]: 2024-05-09 08:58:35.607850453 +0000 UTC m=+0.018326311 image pull d7acac7d56572abea1532ef3a63a1b1ff716d7f1f7a4701680860a80b80cd662 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac 2024-05-09T08:58:35.990 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76346]: 2024-05-09 08:58:35.705287192 +0000 UTC m=+0.115763036 container create b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:35.990 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76346]: 2024-05-09 08:58:35.916418009 +0000 UTC m=+0.326893851 container init b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-09T08:58:35.990 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76346]: 2024-05-09 08:58:35.919639153 +0000 UTC m=+0.330114995 container start b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T08:58:35.990 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:35 smithi188 podman[76346]: 2024-05-09 08:58:35.991442737 +0000 UTC m=+0.401918597 container attach b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True) 2024-05-09T08:58:36.432 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:36 smithi188 podman[76346]: 2024-05-09 08:58:36.210661149 +0000 UTC m=+0.621136994 container died b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T08:58:36.761 INFO:journalctl@ceph.osd.11.smithi188.stdout:May 09 08:58:36 smithi188 podman[76372]: 2024-05-09 08:58:36.744757851 +0000 UTC m=+0.528385976 container remove b1c56e5a9fc6d4b7e88d90bd3584d3b71ad9f2d45bb46e0c200291f59f8d7781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:36.766 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@osd.11.service' 2024-05-09T08:58:36.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:36.804 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T08:58:36.804 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T08:58:36.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.a 2024-05-09T08:58:37.176 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:36 smithi133 systemd[1]: Stopping Ceph mds.a for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:37.552 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: 2024-05-09T08:58:37.359+0000 7f3a1ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:37.552 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:37 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a[70967]: 2024-05-09T08:58:37.359+0000 7f3a1ca00700 -1 mds.a *** got signal Terminated *** 2024-05-09T08:58:39.734 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:39 smithi133 podman[91363]: 2024-05-09 08:58:39.429794527 +0000 UTC m=+2.191414302 container died 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-09T08:58:40.112 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:39 smithi133 podman[91363]: 2024-05-09 08:58:39.85344685 +0000 UTC m=+2.615066624 container cleanup 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-09T08:58:40.112 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:39 smithi133 bash[91363]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a 2024-05-09T08:58:40.427 INFO:journalctl@ceph.mds.a.smithi133.stdout:May 09 08:58:40 smithi133 podman[91388]: 2024-05-09 08:58:40.112803458 +0000 UTC m=+0.666369033 container remove 4300730a79f33318dbe10f1f2c4f6316d888a5388c87a4ef147833b989f96dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-05-09T08:58:40.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.a.service' 2024-05-09T08:58:40.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:40.693 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T08:58:40.693 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T08:58:40.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.d 2024-05-09T08:58:41.176 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:40 smithi133 systemd[1]: Stopping Ceph mds.d for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:41.677 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d[69963]: 2024-05-09T08:58:41.271+0000 7f73c4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:41.678 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:41 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d[69963]: 2024-05-09T08:58:41.271+0000 7f73c4c00700 -1 mds.d *** got signal Terminated *** 2024-05-09T08:58:43.730 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:43 smithi133 podman[91494]: 2024-05-09 08:58:43.467018329 +0000 UTC m=+2.319687521 container died 895e1076ba6560df51fdbd44e6e6a97945eb6e4be50c3aae988039c8741dd5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T08:58:44.091 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:43 smithi133 podman[91494]: 2024-05-09 08:58:43.873505116 +0000 UTC m=+2.726174305 container cleanup 895e1076ba6560df51fdbd44e6e6a97945eb6e4be50c3aae988039c8741dd5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-96805dc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T08:58:44.091 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:43 smithi133 bash[91494]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d 2024-05-09T08:58:44.427 INFO:journalctl@ceph.mds.d.smithi133.stdout:May 09 08:58:44 smithi133 podman[91508]: 2024-05-09 08:58:44.091224594 +0000 UTC m=+0.606079115 container remove 895e1076ba6560df51fdbd44e6e6a97945eb6e4be50c3aae988039c8741dd5de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:58:44.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.d.service' 2024-05-09T08:58:44.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:44.682 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T08:58:44.682 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T08:58:44.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.g 2024-05-09T08:58:44.927 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:44 smithi133 systemd[1]: Stopping Ceph mds.g for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:45.677 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g[70597]: 2024-05-09T08:58:45.207+0000 7fe4b8e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:45.677 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:45 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g[70597]: 2024-05-09T08:58:45.207+0000 7fe4b8e00700 -1 mds.g *** got signal Terminated *** 2024-05-09T08:58:47.677 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:47 smithi133 podman[91613]: 2024-05-09 08:58:47.427282141 +0000 UTC m=+2.334969884 container died 43b3e98c0ccf313950dbf94b42a8ac4c94f890c1d9a92d546ca47a151a80aacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc) 2024-05-09T08:58:48.023 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:47 smithi133 podman[91613]: 2024-05-09 08:58:47.818803242 +0000 UTC m=+2.726491011 container cleanup 43b3e98c0ccf313950dbf94b42a8ac4c94f890c1d9a92d546ca47a151a80aacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc) 2024-05-09T08:58:48.024 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:47 smithi133 bash[91613]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g 2024-05-09T08:58:48.427 INFO:journalctl@ceph.mds.g.smithi133.stdout:May 09 08:58:48 smithi133 podman[91627]: 2024-05-09 08:58:48.019805368 +0000 UTC m=+0.574364690 container remove 43b3e98c0ccf313950dbf94b42a8ac4c94f890c1d9a92d546ca47a151a80aacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-05-09T08:58:48.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.g.service' 2024-05-09T08:58:48.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:48.624 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T08:58:48.624 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T08:58:48.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.j 2024-05-09T08:58:49.177 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:48 smithi133 systemd[1]: Stopping Ceph mds.j for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:49.677 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j[69614]: 2024-05-09T08:58:49.185+0000 7fe268600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:49.677 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:49 smithi133 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j[69614]: 2024-05-09T08:58:49.185+0000 7fe268600700 -1 mds.j *** got signal Terminated *** 2024-05-09T08:58:51.677 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:51 smithi133 podman[91732]: 2024-05-09 08:58:51.416814055 +0000 UTC m=+2.361161949 container died c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T08:58:51.969 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:51 smithi133 podman[91732]: 2024-05-09 08:58:51.830356874 +0000 UTC m=+2.774704751 container cleanup c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T08:58:51.969 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:51 smithi133 bash[91732]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j 2024-05-09T08:58:52.426 INFO:journalctl@ceph.mds.j.smithi133.stdout:May 09 08:58:52 smithi133 podman[91746]: 2024-05-09 08:58:52.098251969 +0000 UTC m=+0.659854794 container remove c4a2d8208e605fb4f4aeff851cc3d6ab29af9f5adb7d1c050b010c00540b75ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:52.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.j.service' 2024-05-09T08:58:52.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:52.715 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T08:58:52.715 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T08:58:52.715 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.b 2024-05-09T08:58:53.034 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:52 smithi160 systemd[1]: Stopping Ceph mds.b for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:53.412 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: 2024-05-09T08:58:53.034+0000 7fe670f5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:53.412 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:53 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b[64462]: 2024-05-09T08:58:53.034+0000 7fe670f5f700 -1 mds.b *** got signal Terminated *** 2024-05-09T08:58:55.633 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:55 smithi160 podman[75947]: 2024-05-09 08:58:55.380991666 +0000 UTC m=+2.468118191 container died 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-96805dc, ceph=True) 2024-05-09T08:58:55.912 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:55 smithi160 podman[75947]: 2024-05-09 08:58:55.734694778 +0000 UTC m=+2.821821294 container cleanup 127928f15eec83096c67377df9d6321f055da657bae690e736ed9267667d9247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T08:58:55.912 INFO:journalctl@ceph.mds.b.smithi160.stdout:May 09 08:58:55 smithi160 bash[75947]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-b 2024-05-09T08:58:56.135 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.b.service' 2024-05-09T08:58:56.168 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:56.169 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T08:58:56.169 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T08:58:56.169 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.e 2024-05-09T08:58:56.478 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:56 smithi160 systemd[1]: Stopping Ceph mds.e for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:58:56.478 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e[63837]: 2024-05-09T08:58:56.479+0000 7f822850b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:58:56.479 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:56 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e[63837]: 2024-05-09T08:58:56.479+0000 7f822850b700 -1 mds.e *** got signal Terminated *** 2024-05-09T08:58:58.870 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:58 smithi160 podman[76069]: 2024-05-09 08:58:58.600465618 +0000 UTC m=+2.216057261 container died 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:58:59.162 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:58 smithi160 podman[76069]: 2024-05-09 08:58:58.954553643 +0000 UTC m=+2.570145283 container cleanup 054fee29c8b02ef6cdc1c52c153fb7f566c9678f89eb10156ad4ac994b6ce783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-09T08:58:59.162 INFO:journalctl@ceph.mds.e.smithi160.stdout:May 09 08:58:58 smithi160 bash[76069]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-e 2024-05-09T08:58:59.374 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.e.service' 2024-05-09T08:58:59.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:58:59.405 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T08:58:59.406 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T08:58:59.406 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.h 2024-05-09T08:58:59.662 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:58:59 smithi160 systemd[1]: Stopping Ceph mds.h for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:00.162 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:58:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: 2024-05-09T08:58:59.751+0000 7fd201aaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:00.162 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:58:59 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h[64149]: 2024-05-09T08:58:59.751+0000 7fd201aaf700 -1 mds.h *** got signal Terminated *** 2024-05-09T08:59:03.662 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:59:03 smithi160 podman[76191]: 2024-05-09 08:59:03.379146976 +0000 UTC m=+3.760981612 container died 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T08:59:04.123 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:59:03 smithi160 podman[76191]: 2024-05-09 08:59:03.868546878 +0000 UTC m=+4.250381518 container cleanup 3af0fcf8920dbdd087cb0d002c55d0e7855dfcd303e98e242ec39c56af316b2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:59:04.123 INFO:journalctl@ceph.mds.h.smithi160.stdout:May 09 08:59:03 smithi160 bash[76191]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-h 2024-05-09T08:59:04.284 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.h.service' 2024-05-09T08:59:04.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:04.314 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T08:59:04.315 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T08:59:04.315 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.k 2024-05-09T08:59:04.662 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:04 smithi160 systemd[1]: Stopping Ceph mds.k for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:04.662 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k[63515]: 2024-05-09T08:59:04.604+0000 7f9ca0f95700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:04.662 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:04 smithi160 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k[63515]: 2024-05-09T08:59:04.605+0000 7f9ca0f95700 -1 mds.k *** got signal Terminated *** 2024-05-09T08:59:07.870 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:07 smithi160 podman[76312]: 2024-05-09 08:59:07.527293157 +0000 UTC m=+3.002856381 container died 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-96805dc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T08:59:08.162 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:07 smithi160 podman[76312]: 2024-05-09 08:59:07.947073155 +0000 UTC m=+3.422636379 container cleanup 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-96805dc) 2024-05-09T08:59:08.162 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:07 smithi160 bash[76312]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k 2024-05-09T08:59:08.162 INFO:journalctl@ceph.mds.k.smithi160.stdout:May 09 08:59:08 smithi160 podman[76326]: 2024-05-09 08:59:08.106459493 +0000 UTC m=+0.581464176 container remove 87c0ed76d2d027c4e28e1b3dfa9cc950dd956df3cb1f76e0a186f97a428d9a87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-k, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-09T08:59:08.318 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.k.service' 2024-05-09T08:59:08.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:08.349 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T08:59:08.349 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T08:59:08.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.c 2024-05-09T08:59:08.649 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:08 smithi188 systemd[1]: Stopping Ceph mds.c for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:08.909 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: 2024-05-09T08:59:08.649+0000 7f18413f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:08.909 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:08 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c[65500]: 2024-05-09T08:59:08.649+0000 7f18413f7700 -1 mds.c *** got signal Terminated *** 2024-05-09T08:59:11.682 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:11 smithi188 podman[76452]: 2024-05-09 08:59:11.333594104 +0000 UTC m=+2.785988861 container died d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-96805dc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T08:59:12.066 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:11 smithi188 podman[76452]: 2024-05-09 08:59:11.841787586 +0000 UTC m=+3.294182340 container cleanup d3ceadf87dfb0f3e3feb1bb40d0dd676051dfaa8ca3303f4c2e9b663e742eb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=squid-96805dc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T08:59:12.066 INFO:journalctl@ceph.mds.c.smithi188.stdout:May 09 08:59:11 smithi188 bash[76452]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-c 2024-05-09T08:59:12.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.c.service' 2024-05-09T08:59:12.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:12.313 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T08:59:12.313 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T08:59:12.313 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.f 2024-05-09T08:59:12.682 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:12 smithi188 systemd[1]: Stopping Ceph mds.f for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:12.682 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f[65192]: 2024-05-09T08:59:12.620+0000 7fa28d166700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:12.682 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:12 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f[65192]: 2024-05-09T08:59:12.620+0000 7fa28d166700 -1 mds.f *** got signal Terminated *** 2024-05-09T08:59:16.932 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:16 smithi188 podman[76586]: 2024-05-09 08:59:16.642793836 +0000 UTC m=+4.117063876 container died f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T08:59:17.280 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:16 smithi188 podman[76586]: 2024-05-09 08:59:16.95333977 +0000 UTC m=+4.427609818 container cleanup f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:59:17.280 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:16 smithi188 bash[76586]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f 2024-05-09T08:59:17.280 INFO:journalctl@ceph.mds.f.smithi188.stdout:May 09 08:59:17 smithi188 podman[76600]: 2024-05-09 08:59:17.171051786 +0000 UTC m=+0.526722229 container remove f0ec3fa00b3faca809f7b4df81b9c6d07d9ce71ca60d5041859d8c9854502d5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-96805dc, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T08:59:17.291 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.f.service' 2024-05-09T08:59:17.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:17.324 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T08:59:17.324 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T08:59:17.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.i 2024-05-09T08:59:17.647 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:17 smithi188 systemd[1]: Stopping Ceph mds.i for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:17.931 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: 2024-05-09T08:59:17.647+0000 7f24d816c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:17.932 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:17 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i[65806]: 2024-05-09T08:59:17.647+0000 7f24d816c700 -1 mds.i *** got signal Terminated *** 2024-05-09T08:59:19.631 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:19 smithi188 podman[76705]: 2024-05-09 08:59:19.338170427 +0000 UTC m=+1.794564496 container died 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-96805dc, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T08:59:19.898 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:19 smithi188 podman[76705]: 2024-05-09 08:59:19.731706973 +0000 UTC m=+2.188101036 container cleanup 75e13b9ccea1f683a20eca64c0e76b187893342d33f2f18996ed00d0a8a7bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-09T08:59:19.898 INFO:journalctl@ceph.mds.i.smithi188.stdout:May 09 08:59:19 smithi188 bash[76705]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-i 2024-05-09T08:59:20.115 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.i.service' 2024-05-09T08:59:20.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:20.145 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T08:59:20.145 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T08:59:20.145 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.l 2024-05-09T08:59:20.682 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:20 smithi188 systemd[1]: Stopping Ceph mds.l for 99f031e6-0ddc-11ef-bc98-c7b262605968... 2024-05-09T08:59:20.682 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l[64885]: 2024-05-09T08:59:20.451+0000 7f4706eab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T08:59:20.682 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:20 smithi188 ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l[64885]: 2024-05-09T08:59:20.451+0000 7f4706eab700 -1 mds.l *** got signal Terminated *** 2024-05-09T08:59:24.822 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:24 smithi188 podman[76824]: 2024-05-09 08:59:24.457672286 +0000 UTC m=+4.097412344 container died acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-96805dc, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-09T08:59:25.148 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:24 smithi188 podman[76824]: 2024-05-09 08:59:24.898669545 +0000 UTC m=+4.538409605 container cleanup acb782d122e645b0455812611c99d6a0ebaa937396868698fc14b026be24cdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3b573cb0f3e3b36c210f25843c0a0bb7a496943c2eb5ca04944a503c246a55ac, name=ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-96805dc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T08:59:25.148 INFO:journalctl@ceph.mds.l.smithi188.stdout:May 09 08:59:24 smithi188 bash[76824]: ceph-99f031e6-0ddc-11ef-bc98-c7b262605968-mds-l 2024-05-09T08:59:25.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-99f031e6-0ddc-11ef-bc98-c7b262605968@mds.l.service' 2024-05-09T08:59:25.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T08:59:25.386 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T08:59:25.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T08:59:25.711 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:59:30.691 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T08:59:30.879 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:59:33.506 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-09T08:59:33.696 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T08:59:36.087 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:36.182 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:36.213 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T08:59:36.240 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T08:59:36.242 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi133/crash 2024-05-09T08:59:36.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash -- . 2024-05-09T08:59:36.348 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T08:59:36.348 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-05-09T08:59:36.353 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi160/crash 2024-05-09T08:59:36.354 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash -- . 2024-05-09T08:59:36.390 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T08:59:36.390 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2024-05-09T08:59:36.392 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi188/crash 2024-05-09T08:59:36.392 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash -- . 2024-05-09T08:59:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T08:59:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-05-09T08:59:36.431 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T08:59:36.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-09T08:59:36.581 INFO:tasks.cephadm:Compressing logs... 2024-05-09T08:59:36.582 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:59:36.587 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:59:36.589 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:: No such file or directory 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-09T08:59:36.617 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.i.log: /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr: 83.3%/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.11.log: -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr: 90.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T08:59:36.618 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log: 86.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T08:59:36.620 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T08:59:36.620 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T08:59:36.620 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log 2024-05-09T08:59:36.628 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T08:59:36.628 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-09T08:59:36.628 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-09T08:59:36.628 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-09T08:59:36.629 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-09T08:59:36.629 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-09T08:59:36.629 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-09T08:59:36.629 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-09T08:59:36.629 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T08:59:36.630 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.h.log: /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-09T08:59:36.630 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T08:59:36.630 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-09T08:59:36.630 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-09T08:59:36.631 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-09T08:59:36.681 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T08:59:36.686 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T08:59:36.686 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T08:59:36.687 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log 2024-05-09T08:59:36.688 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-09T08:59:36.690 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-09T08:59:36.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-09T08:59:36.692 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-09T08:59:36.693 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-09T08:59:36.696 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-09T08:59:36.696 INFO:teuthology.orchestra.run.smithi133.stderr: 86.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T08:59:36.696 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-09T08:59:36.697 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-09T08:59:36.698 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-09T08:59:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip 94.5% -5 -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.j.log.gz --verbose 2024-05-09T08:59:36.699 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-09T08:59:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-09T08:59:36.701 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-09T08:59:36.702 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-09T08:59:36.704 INFO:teuthology.orchestra.run.smithi188.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T08:59:36.708 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.d.log: /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-09T08:59:36.731 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.b.log: /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.k.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T08:59:36.777 INFO:teuthology.orchestra.run.smithi133.stderr: 83.2% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T08:59:36.800 INFO:teuthology.orchestra.run.smithi160.stderr: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-09T08:59:36.808 INFO:teuthology.orchestra.run.smithi133.stderr: 90.2% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T08:59:36.818 INFO:teuthology.orchestra.run.smithi160.stderr: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-09T08:59:36.865 INFO:teuthology.orchestra.run.smithi188.stderr: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-09T08:59:36.902 INFO:teuthology.orchestra.run.smithi133.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T08:59:36.916 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T08:59:36.931 INFO:teuthology.orchestra.run.smithi160.stderr: 83.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-09T08:59:36.939 INFO:teuthology.orchestra.run.smithi160.stderr: 90.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-09T08:59:36.980 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T08:59:37.027 INFO:teuthology.orchestra.run.smithi160.stderr: 93.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T08:59:37.059 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-09T08:59:37.109 INFO:teuthology.orchestra.run.smithi188.stderr: 92.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-09T08:59:37.159 INFO:teuthology.orchestra.run.smithi160.stderr: 86.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-09T08:59:39.027 INFO:teuthology.orchestra.run.smithi133.stderr: 90.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T08:59:44.149 INFO:teuthology.orchestra.run.smithi160.stderr: 92.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-09T08:59:46.019 INFO:teuthology.orchestra.run.smithi188.stderr: 95.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-09T09:00:05.972 INFO:teuthology.orchestra.run.smithi188.stderr: 94.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-09T09:00:15.386 INFO:teuthology.orchestra.run.smithi133.stderr: 92.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-09T09:00:23.244 INFO:teuthology.orchestra.run.smithi188.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-09T09:00:24.700 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-09T09:00:34.187 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-09T09:00:37.729 INFO:teuthology.orchestra.run.smithi188.stderr: 90.9% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-09T09:00:37.733 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-05-09T09:00:37.733 INFO:teuthology.orchestra.run.smithi188.stderr:real 1m1.129s 2024-05-09T09:00:37.733 INFO:teuthology.orchestra.run.smithi188.stderr:user 3m43.079s 2024-05-09T09:00:37.733 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m6.869s 2024-05-09T09:03:35.330 INFO:teuthology.orchestra.run.smithi160.stderr: 95.4% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-09T09:05:05.664 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-09T09:05:13.757 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-09T09:05:41.690 INFO:teuthology.orchestra.run.smithi133.stderr: 91.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-09T09:06:55.659 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-09T09:07:14.951 INFO:teuthology.orchestra.run.smithi160.stderr: 91.3% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-09T09:08:13.516 INFO:teuthology.orchestra.run.smithi160.stderr: 91.6% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-09T09:09:39.978 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-09T09:10:16.680 INFO:teuthology.orchestra.run.smithi133.stderr: 93.5% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-09T09:10:39.174 INFO:teuthology.orchestra.run.smithi160.stderr: 91.1% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-09T09:10:51.938 INFO:teuthology.orchestra.run.smithi133.stderr: 93.3% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-09T09:10:51.942 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-05-09T09:10:51.942 INFO:teuthology.orchestra.run.smithi133.stderr:real 11m15.315s 2024-05-09T09:10:51.942 INFO:teuthology.orchestra.run.smithi133.stderr:user 13m17.847s 2024-05-09T09:10:51.942 INFO:teuthology.orchestra.run.smithi133.stderr:sys 1m38.830s 2024-05-09T09:18:34.820 INFO:teuthology.orchestra.run.smithi160.stderr: 93.0% -- replaced with /var/log/ceph/99f031e6-0ddc-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-09T09:18:35.151 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T09:18:35.152 INFO:teuthology.orchestra.run.smithi160.stderr:real 18m58.267s 2024-05-09T09:18:35.152 INFO:teuthology.orchestra.run.smithi160.stderr:user 10m34.161s 2024-05-09T09:18:35.152 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m27.011s 2024-05-09T09:18:35.152 INFO:tasks.cephadm:Archiving logs... 2024-05-09T09:18:35.154 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi133/log 2024-05-09T09:18:35.155 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:19:45.022 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi160/log 2024-05-09T09:19:45.023 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:21:29.928 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi188/log 2024-05-09T09:21:29.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T09:22:06.413 INFO:tasks.cephadm:Removing cluster... 2024-05-09T09:22:06.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force 2024-05-09T09:22:07.309 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T09:22:09.101 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force 2024-05-09T09:22:09.931 INFO:teuthology.orchestra.run.smithi160.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T09:22:10.743 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 99f031e6-0ddc-11ef-bc98-c7b262605968 --force 2024-05-09T09:22:10.936 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 99f031e6-0ddc-11ef-bc98-c7b262605968 2024-05-09T09:22:11.482 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T09:22:11.482 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:22:11.540 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:22:11.558 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T09:22:11.574 INFO:tasks.cephadm:Teardown complete 2024-05-09T09:22:11.574 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T09:22:11.589 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T09:22:11.589 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:22:11.597 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:22:11.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T09:22:11.750 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T09:22:11.764 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T09:22:11.764 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:22:11.772 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:22:11.774 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T09:22:11.786 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:22:11.788 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:22:11.801 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:22:11.802 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-09T09:22:11.802 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 56 +488us[ +511us] +/- 69ms 2024-05-09T09:22:11.802 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -760us[ -736us] +/- 60ms 2024-05-09T09:22:11.802 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +409us[ +409us] +/- 62ms 2024-05-09T09:22:11.802 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:22:11.819 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 51 +439us[ +438us] +/- 69ms 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -787us[ -788us] +/- 60ms 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 57 +401us[ +400us] +/- 62ms 2024-05-09T09:22:11.868 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:22:11.923 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T09:22:11.923 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-05-09T09:22:11.923 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +469us[ +484us] +/- 69ms 2024-05-09T09:22:11.924 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -758us[ -743us] +/- 60ms 2024-05-09T09:22:11.924 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +404us[ +419us] +/- 62ms 2024-05-09T09:22:11.924 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T09:22:11.927 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T09:22:11.938 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T09:22:11.939 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T09:22:11.951 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:22:12.224 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:22:12.344 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T09:22:12.428 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:22:12.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:12.521 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:22:12.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:12.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T09:22:12.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:12.579 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T09:22:12.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T09:22:12.633 INFO:teuthology.task.internal:Duration was 4339.420840 seconds 2024-05-09T09:22:12.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T09:22:12.653 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T09:22:12.653 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:22:12.661 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:22:12.664 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T09:22:12.719 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:22:12.724 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:22:12.815 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T09:22:13.291 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T09:22:13.291 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-05-09T09:22:13.291 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:22:13.358 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T09:22:13.358 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:22:13.390 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T09:22:13.390 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T09:22:13.416 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T09:22:13.416 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:22:13.424 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:22:13.433 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T09:22:13.505 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T09:22:13.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:22:13.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:22:13.513 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T09:22:14.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T09:22:14.049 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T09:22:14.049 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:22:14.132 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:22:14.158 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T09:22:14.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T09:22:14.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.210 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.212 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.245 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-05-09T09:22:14.251 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-09T09:22:14.320 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-05-09T09:22:14.369 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:14.417 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:14.432 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T09:22:14.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T09:22:14.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T09:22:14.468 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T09:22:14.469 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi133 2024-05-09T09:22:14.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:22:14.575 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi160 2024-05-09T09:22:14.575 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:22:14.611 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698663/remote/smithi188 2024-05-09T09:22:14.611 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T09:22:14.650 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T09:22:14.651 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:22:14.658 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:22:14.661 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T09:22:14.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T09:22:14.722 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T09:22:14.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T09:22:14.733 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T09:22:14.733 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:22:14.739 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:22:14.741 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T09:22:14.753 INFO:teuthology.orchestra.run.smithi160.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:22 /home/ubuntu/cephtest 2024-05-09T09:22:14.759 INFO:teuthology.orchestra.run.smithi188.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:22 /home/ubuntu/cephtest 2024-05-09T09:22:14.789 INFO:teuthology.orchestra.run.smithi133.stdout: 393472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 09:22 /home/ubuntu/cephtest 2024-05-09T09:22:14.795 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T09:22:14.810 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T09:22:14.847 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/dbench}} duration: 4339.420840024948 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T09:22:14.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T09:22:14.946 INFO:teuthology.run:pass